Pump it Up: Data Mining the Water Table¶

Analysed By : Osafa Karim (osafa.karim@gmail.com)¶

Analysed On : 4th July 2019¶

Analysed for : Axiata Analytics - Employment Test¶

Python kernel Used¶

# kernel used
from platform import python_version
print('The version of ',python_version())

The version of  3.6.8

# importing the data-analysis and visualization libraries
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
from datetime import datetime 
%matplotlib inline

import os
import requests

import re 
import geopy.distance

pd.set_option('display.max_columns', None)
pd.set_option('display.max_rows', None)

Download the data set for analyses from the url as is in the¶

https://www.drivendata.org/competitions/7/pump-it-up-data-mining-the-water-table/data/¶

def download_data(url, file_name):
    '''
    input : 
       1. url - the url from where the file has to be downloaded
       2. file_name - the name with with the corresponding file has to be created 
    '''
    if file_name not in os.listdir():
        t0 = datetime.now()
        response = requests.get(url)
        with open(os.path.join(os.getcwd(), file_name), 'wb') as f:
            f.write(response.content)
        
        print('****** File ', file_name,', of size :',os.path.getsize(file_name)/1000000 ,' MB sucessfully downloaded in ',
             str((datetime.now()- t0).seconds),' seconds !!!!.')
    else:
        print('****** File with name ',file_name,' already exists. Please verify.')

# training-set values
url = 'https://s3.amazonaws.com/drivendata/data/7/public/4910797b-ee55-40a7-8668-10efd5c1b960.csv'
file_name = 'raw_train_feature_vectors.csv'
download_data(url, file_name)

****** File with name  raw_train_feature_vectors.csv  already exists. Please verify.

# training-set labels
url = 'https://s3.amazonaws.com/drivendata/data/7/public/0bf8bc6e-30d0-4c50-956a-603fc693d966.csv'
file_name = 'raw_train_target_vectors.csv'
download_data(url, file_name)

****** File with name  raw_train_target_vectors.csv  already exists. Please verify.

# testing-set values
url = 'https://s3.amazonaws.com/drivendata/data/7/public/702ddfc5-68cd-4d1d-a0de-f5f566f76d91.csv'
file_name = 'raw_test_feature_vectors.csv'
download_data(url, file_name)

****** File with name  raw_test_feature_vectors.csv  already exists. Please verify.

Data Description¶

# train data-set 

train_data = pd.read_csv('raw_train_feature_vectors.csv')

print('There {} features and {} data-points in train-data set.'.format(train_data.shape[1],train_data.shape[0]))

There 40 features and 59400 data-points in train-data set.

# get the data-types of the features
train_data.info()

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 59400 entries, 0 to 59399
Data columns (total 40 columns):
id                       59400 non-null int64
amount_tsh               59400 non-null float64
date_recorded            59400 non-null object
funder                   55765 non-null object
gps_height               59400 non-null int64
installer                55745 non-null object
longitude                59400 non-null float64
latitude                 59400 non-null float64
wpt_name                 59400 non-null object
num_private              59400 non-null int64
basin                    59400 non-null object
subvillage               59029 non-null object
region                   59400 non-null object
region_code              59400 non-null int64
district_code            59400 non-null int64
lga                      59400 non-null object
ward                     59400 non-null object
population               59400 non-null int64
public_meeting           56066 non-null object
recorded_by              59400 non-null object
scheme_management        55523 non-null object
scheme_name              31234 non-null object
permit                   56344 non-null object
construction_year        59400 non-null int64
extraction_type          59400 non-null object
extraction_type_group    59400 non-null object
extraction_type_class    59400 non-null object
management               59400 non-null object
management_group         59400 non-null object
payment                  59400 non-null object
payment_type             59400 non-null object
water_quality            59400 non-null object
quality_group            59400 non-null object
quantity                 59400 non-null object
quantity_group           59400 non-null object
source                   59400 non-null object
source_type              59400 non-null object
source_class             59400 non-null object
waterpoint_type          59400 non-null object
waterpoint_type_group    59400 non-null object
dtypes: float64(3), int64(7), object(30)
memory usage: 18.1+ MB

# Casting the data types of date fileds to datetime dtype
train_data['date_recorded'] = pd.to_datetime(train_data['date_recorded'])
#train_data['construction_year'] = pd.to_datetime(train_data['construction_year'])

Check for missing data¶

#train_data.isna().sum()
for a in train_data.columns[train_data.isna().any()].tolist():
    print(a ,' has ',train_data[train_data[a].isna()].shape[0],' missing values')

funder  has  3635  missing values
installer  has  3655  missing values
subvillage  has  371  missing values
public_meeting  has  3334  missing values
scheme_management  has  3877  missing values
scheme_name  has  28166  missing values
permit  has  3056  missing values

# Count of types of features
categorical = len(train_data.select_dtypes(include='object').columns)
integ = len(train_data.select_dtypes(include='int64').columns)
float_num = len(train_data.select_dtypes(include='float64').columns)
date_time = len(train_data.select_dtypes(include='datetime64[ns]').columns)

categorical, integ, float_num, date_time

(29, 7, 3, 1)

dty_list =[]

for a, b in zip([categorical, integ, float_num, date_time],['categorical', 'integ', 'float_num', 'date_time']):
    for _ in range(a):
            dty_list.append(b)

plt.hist(dty_list, color = 'green')
plt.xlabel('\n Data types of the features')
plt.ylabel('Counts')
plt.title(' Comparison of the fetaures by dtypes')

Text(0.5, 1.0, ' Comparison of the fetaures by dtypes')

# Start dissecting the categorical features 
[ col for col in train_data.columns if train_data[col].dtypes == 'object']

['funder',
 'installer',
 'wpt_name',
 'basin',
 'subvillage',
 'region',
 'lga',
 'ward',
 'public_meeting',
 'recorded_by',
 'scheme_management',
 'scheme_name',
 'permit',
 'extraction_type',
 'extraction_type_group',
 'extraction_type_class',
 'management',
 'management_group',
 'payment',
 'payment_type',
 'water_quality',
 'quality_group',
 'quantity',
 'quantity_group',
 'source',
 'source_type',
 'source_class',
 'waterpoint_type',
 'waterpoint_type_group']

Let's visualize and analyze each of the categorical features for more insights .¶

1. Funder - Let's visualize the top 30 Organizations who have funded the water-resources¶

fund_30 = pd.DataFrame(train_data['funder'].value_counts().head(30))

ax = fund_30.T.plot(kind='bar', figsize = (18, 10) , grid = True )
for p in ax.patches:
    ax.annotate(str(p.get_height()), (p.get_x() * 1.005, p.get_height() * 1.005))

2. Installer - Let's visualize top 30 installers¶

install_30 = pd.DataFrame(train_data['installer'].value_counts().head(30))

ax = install_30.T.plot(kind = 'bar' , figsize = (16, 10))
for p in ax.patches:
     ax.annotate(str(p.get_height()), (p.get_x() * 1.005, p.get_height() * 1.005))

*** Insight -1 ; funders and installers may or may not be same.¶

3. wpt_name - The name of the water points in the locality¶

wpt_30  = pd.DataFrame(train_data['wpt_name'].value_counts().head(30))

ax = wpt_30.T.plot(kind = 'bar' , figsize = (17, 10))
for p in ax.patches:
    ax.annotate(str(p.get_height()), (p.get_x() * 1.005, p.get_height() * 1.005))

*** Insight -2 ; A good number of locations doesn't have water_points¶

4. basin - Geographical Water Basin¶

len(train_data['basin'].unique())

9

# checking if there are missing values for basin
train_data['basin'].isna().sum()

0

*** Insight -3 . There are 9 basins from where all the waters are supplied.¶

# Let's visualize , how basins compare in terms of supplying the water
basin_df = pd.DataFrame(train_data['basin'].value_counts())

ax = basin_df.T.plot(kind = 'bar', figsize=(17,7))
for p in ax.patches:
    ax.annotate(str(p.get_height()), (p.get_x() * 1.005, p.get_height() * 1.005))

5. subvillage - Geographical Region¶

#train_data['subvillage'].value_counts()

train_data['subvillage'].isna().sum()

371

len(train_data['subvillage'].unique())

19288

At this point there is an intuition that all the features related geographical distance has a bearance in terms of proximity to main-water-bidies(basins) , which are the source of supplies for these water-points.¶

'Lake Victoria' = Latitude: -1° 00' 0.00" S Longitude: 33° 00' 0.00" E

'Pangani' = Latitude: -5° 25' 30.94" S Longitude: 38° 58' 29.03" E

'Rufiji' = Latitude: -8° 00' 0.00" S Longitude: 39° 19' 60.00" E

'Internal'(same as ruvu) = Latitude: -6° 48' 17.99" S Longitude: 38° 41' 59.99" E

'Lake Tanganyika' = 6.2556° S, 29.5108° E

'Wami' - Latitude: -6° 07' 60.00" S Longitude: 38° 48' 59.99" E

'Lake Nyasa' = 11.6701° S, 34.6857° E

'Ruvuma' = 10.6879° S, 36.2631° E

'Lake Rukwa' = 8.0167° S, 32.2654° E

train_data['basin'].value_counts()

Lake Victoria              10248
Pangani                     8940
Rufiji                      7976
Internal                    7785
Lake Tanganyika             6432
Wami / Ruvu                 5987
Lake Nyasa                  5085
Ruvuma / Southern Coast     4493
Lake Rukwa                  2454
Name: basin, dtype: int64

6. region - Geographic location (coded)¶

train_data['region'].value_counts()

Iringa           5294
Shinyanga        4982
Mbeya            4639
Kilimanjaro      4379
Morogoro         4006
Arusha           3350
Kagera           3316
Mwanza           3102
Kigoma           2816
Ruvuma           2640
Pwani            2635
Tanga            2547
Dodoma           2201
Singida          2093
Mara             1969
Tabora           1959
Rukwa            1808
Mtwara           1730
Manyara          1583
Lindi            1546
Dar es Salaam     805
Name: region, dtype: int64

region_data = pd.DataFrame(train_data['region'].value_counts())

ax = region_data.T.plot(kind = 'bar', figsize= (16,9))
for p in ax.patches:
    ax.annotate(str(p.get_height()), (p.get_x() * 1.005, p.get_height() * 1.005))

7.lga - Geographic location¶

print('Number of LGA in the dataset : ',len(train_data['lga'].unique()),
      ' and number of missing entries is ',train_data['lga'].isna().sum())

Number of LGA in the dataset :  125  and number of missing entries is  0

print('Number of regions in the dataset : ',len(train_data['region'].unique()),
     ' and number of missing entries is ',train_data['region'].isna().sum())

Number of regions in the dataset :  21  and number of missing entries is  0

*** Insight 4 - lga and region might be redundant informations for predictive modelling, for analysis and administrative view of which locations and their corresponding sub-locations has more issues¶

agg_reg_lga = train_data[['region','lga']].groupby('region').count()

ax = agg_reg_lga.T.plot(kind = 'bar', figsize=(17, 10))
for p in ax.patches:
    ax.annotate(str(p.get_height()), (p.get_x() * 1.005, p.get_height() * 1.005))
plt.title(' Regions by number of sub-regions ')
plt.ylabel(' Count of lga ')

Text(0, 0.5, ' Count of lga ')

8. ward - Geographic location¶

print('Number of wards in the dataset : ',len(train_data['ward'].unique()),
     ' and number of missing entries is ',train_data['ward'].isna().sum())

Number of wards in the dataset :  2092  and number of missing entries is  0

** Insight - 5 So "ward" is the sub-category of the "lga" which is sub-category to "region"¶

location_df = train_data[['region','lga','ward']].groupby(['region','lga']).count()

location_df

9. public_meeting - True/False¶

train_data['public_meeting'].value_counts()

True     51011
False     5055
Name: public_meeting, dtype: int64

ax = pd.DataFrame(train_data['public_meeting'].value_counts()).T.plot(kind='bar')
for p in ax.patches:
    ax.annotate(str(p.get_height()), (p.get_x() * 1.007, p.get_height() * 1.007))

10. recorded_by - Group entering this row of data¶

train_data['recorded_by'].value_counts()

GeoData Consultants Ltd    59400
Name: recorded_by, dtype: int64

** All the data was recorded by "GeoData Consultants Ltd"¶

This info may be left out while model creation as there isn't anything a recorder can do to make prediction of the pumps condition.¶

11. scheme_management - Who operates the waterpoint¶

scheme_mgmt = pd.DataFrame(train_data['scheme_management'].value_counts())

ax = scheme_mgmt.T.plot(kind = 'bar', figsize=(18,9) , colormap = 'Paired', use_index= False)
for p in ax.patches:
    ax.annotate(str(p.get_height()), (p.get_x() * 1.007, p.get_height() * 1.007))
plt.xlabel(' Scheme Management')

Text(0.5, 0, ' Scheme Management')

train_data['scheme_management'].isna().sum()

3877

12. scheme_name - Who operates the waterpoint¶

Hence, the above figure . The scheme_management is useful and scheme_name becomes redundant info.¶

13. permit - If the waterpoint is permitted¶

permit_df = pd.DataFrame(train_data['permit'].value_counts())

ax = permit_df.T.plot(kind = 'bar', use_index = False )
for p in ax.patches:
    ax.annotate(str(p.get_height()), (p.get_x() * 1.007, p.get_height() * 1.007))
plt.xlabel(' Permit')

Text(0.5, 0, ' Permit')

*** Insight 6 - A huge number of pumps , nearly a third are without permit.¶

14. extraction_type - The kind of extraction the waterpoint uses¶

extract_df = pd.DataFrame(train_data['extraction_type'].value_counts())

ax = extract_df.T.plot(kind = 'bar', figsize=(17, 8), use_index=False)
for p in ax.patches:
    ax.annotate(str(p.get_height()), (p.get_x() * 1.007, p.get_height() * 1.007))
plt.xlabel(' Extraction Type')

Text(0.5, 0, ' Extraction Type')

*** Insight 7 - The highest , about 50 percent of the pumps use gravity as extraction_type.¶

15. extraction_type_group - The kind of extraction the waterpoint uses¶

train_data['extraction_type_group'].value_counts()

gravity            26780
nira/tanira         8154
other               6430
submersible         6179
swn 80              3670
mono                2865
india mark ii       2400
afridev             1770
rope pump            451
other handpump       364
other motorpump      122
wind-powered         117
india mark iii        98
Name: extraction_type_group, dtype: int64

train_data['extraction_type'].value_counts()

gravity                      26780
nira/tanira                   8154
other                         6430
submersible                   4764
swn 80                        3670
mono                          2865
india mark ii                 2400
afridev                       1770
ksb                           1415
other - rope pump              451
other - swn 81                 229
windmill                       117
india mark iii                  98
cemo                            90
other - play pump               85
walimi                          48
climax                          32
other - mkulima/shinyanga        2
Name: extraction_type, dtype: int64

train_data['extraction_type_group'].isna().sum()

0

train_data['extraction_type'].isna().sum()

0

Both "extraction_type_group" and "extraction_type" are all but same , hence we can use either and drop the other for predictive modelling.¶

16. extraction_type_class - The kind of extraction the waterpoint uses¶

train_data['extraction_type_class'].value_counts()

gravity         26780
handpump        16456
other            6430
submersible      6179
motorpump        2987
rope pump         451
wind-powered      117
Name: extraction_type_class, dtype: int64

train_data['extraction_type_class'].isna().sum()

0

17. management - How the waterpoint is managed¶

mgmt_df = pd.DataFrame(train_data['management'].value_counts())

ax = mgmt_df.T.plot(kind = 'bar', figsize=(16,8), use_index=False)
for p in ax.patches:
    ax.annotate(str(p.get_height()), (p.get_x() * 1.007, p.get_height() * 1.007))
plt.xlabel(' Management')

Text(0.5, 0, ' Management')

18. management_group - How the waterpoint is managed¶

mgmt_grp_df = pd.DataFrame(train_data['management_group'].value_counts())

ax = mgmt_grp_df.T.plot(kind = 'bar', figsize=(17, 8), use_index=False)
for p in ax.patches:
    for p in ax.patches:
        ax.annotate(str(p.get_height()), (p.get_x() * 1.007, p.get_height() * 1.007))
plt.xlabel(' Management Group')

Text(0.5, 0, ' Management Group')

19. payment - What the water costs¶

pay_df = pd.DataFrame(train_data['payment'].value_counts())

ax = pay_df.T.plot(kind = 'bar', figsize=(16, 7), use_index=False)
for p in ax.patches:
    for p in ax.patches:
        ax.annotate(str(p.get_height()), (p.get_x() * 1.007, p.get_height() * 1.007))
plt.xlabel(' Payment')

Text(0.5, 0, ' Payment')

*** Insight 8 - There are huge no-payments , gives the intuitions of what maintenance would have been done.¶

20. payment_type - What the water costs¶

train_data['payment_type'].value_counts()

never pay     25348
per bucket     8985
monthly        8300
unknown        8157
on failure     3914
annually       3642
other          1054
Name: payment_type, dtype: int64

train_data['payment'].value_counts()

never pay                25348
pay per bucket            8985
pay monthly               8300
unknown                   8157
pay when scheme fails     3914
pay annually              3642
other                     1054
Name: payment, dtype: int64

"Payment_type" and "Payment" are duplicate , hence one has to dropped while modelling.¶

21. water_quality - The quality of the water¶

wtr_qlt_df = pd.DataFrame(train_data['water_quality'].value_counts())

ax = wtr_qlt_df.T.plot(kind = 'bar', figsize=(16,8), use_index=False)
for p in ax.patches:
    ax.annotate(str(p.get_height()), (p.get_x() * 1.007, p.get_height() * 1.007))
plt.xlabel(' Water Quality ')

Text(0.5, 0, ' Water Quality ')

*** Insight 9 - Most of the water are soft .¶

22. quality_group - The quality of the water¶

qlt_grp_df = pd.DataFrame(train_data['quality_group'].value_counts())

ax = qlt_grp_df.T.plot(kind = 'bar', figsize=(16,8), use_index=False)
for p in ax.patches:
    ax.annotate(str(p.get_height()), (p.get_x() * 1.007, p.get_height() * 1.007))
plt.xlabel(' Quality Group ')

Text(0.5, 0, ' Quality Group ')

"water_quality" and "quality_group" apparently from the above 2 figures look same/redundant , hence one can drop either one while modelling.¶

23. quantity - The quantity of water¶

train_data['quantity'].value_counts()

enough          33186
insufficient    15129
dry              6246
seasonal         4050
unknown           789
Name: quantity, dtype: int64

24. quantity_group - The quantity of water¶

train_data['quantity_group'].value_counts()

enough          33186
insufficient    15129
dry              6246
seasonal         4050
unknown           789
Name: quantity_group, dtype: int64

"quantity" and "quantity_group" are redundant hence , one can drop either and just use one for modelling.¶

25. source - The source of the water¶

train_data['source'].value_counts()

spring                  17021
shallow well            16824
machine dbh             11075
river                    9612
rainwater harvesting     2295
hand dtw                  874
lake                      765
dam                       656
other                     212
unknown                    66
Name: source, dtype: int64

26. source_type - The source of the water¶

train_data['source_type'].value_counts()

spring                  17021
shallow well            16824
borehole                11949
river/lake              10377
rainwater harvesting     2295
dam                       656
other                     278
Name: source_type, dtype: int64

'Source_type' looks redundant to "source" hence , while modelling one can drop "source_type".¶

27. source_class - The source of the water¶

src_cls_df = pd.DataFrame(train_data['source_class'].value_counts())

ax = src_cls_df.T.plot(kind = 'bar', figsize=(16,8), use_index=False)
for p in ax.patches:
    ax.annotate(str(p.get_height()), (p.get_x() * 1.007, p.get_height() * 1.007))
plt.xlabel(' Source Class ')

Text(0.5, 0, ' Source Class ')

28. waterpoint_type - The kind of waterpoint¶

train_data['waterpoint_type'].value_counts()

communal standpipe             28522
hand pump                      17488
other                           6380
communal standpipe multiple     6103
improved spring                  784
cattle trough                    116
dam                                7
Name: waterpoint_type, dtype: int64

29 . waterpoint_type_group - The kind of waterpoint¶

train_data['waterpoint_type_group'].value_counts()

communal standpipe    34625
hand pump             17488
other                  6380
improved spring         784
cattle trough           116
dam                       7
Name: waterpoint_type_group, dtype: int64

"waterpoint_type" and "waterpoint_type_group" are redundant and either one can be dropped while modelling.¶

Let's visualize and analyze each of the numerical features for more insights .¶

[col for col in train_data.columns if train_data[col].dtypes in ('int64','float64')]

['id',
 'amount_tsh',
 'gps_height',
 'longitude',
 'latitude',
 'num_private',
 'region_code',
 'district_code',
 'population',
 'construction_year']

1. Id is just id , hence ignore it¶

2. amount_tsh - Total static head (amount water available to waterpoint)¶

tsh_df = train_data[train_data['amount_tsh']  != 0]['amount_tsh']

len(tsh_df)

17761

len(train_data[train_data['amount_tsh'] == 0])

41639

tsh_df.hist(bins = 500)
plt.xlim([0, 10000])

(0, 10000)

'Total static head' feature is higly skewd(right) with 41693 (70% of records) equal to zero. Hence this has to be imputated using some technique (mean or median).¶

3. gps_height - Altitude of the well¶

train_data['gps_height'].isna().sum()

0

train_data['gps_height'].hist(bins = 10)

<matplotlib.axes._subplots.AxesSubplot at 0x1a276d0d30>

*** Insight 10 - The "gps height" is also skewed , meaning that wells are generally built at a certain heigh in Tanzania.¶

4. longitude - GPS coordinate . This will be dealt separately , as it will be used to calculate the distance from the basin.¶

5. latitude - GPS coordinate. This will be dealt separately , as it will be used to calculate the distance from the basin.¶

6. num_private - The info is completely missing hence nothing can be said or hypothesissed.¶

7. region_code - Geographic location (coded) - Locations are important to determine the proximity to the main water bodies , but we will use gps coordinates for the same.¶

8. district_code - Geographic location (coded) - Locations are important to determine the proximity to the main water bodies , but we will use gps coordinates for the same.¶

9. population - population - Population around the well¶

train_data['population'].isna().sum()

0

plt.figure(figsize=(17,8))
train_data['population'].hist(bins = 1000)
plt.xlim([0, 3000])

(0, 3000)

train_data['population'].max()

30500

plt.figure(figsize=(17,10))
plt.grid(True)
sns.distplot(train_data['population'] , bins = 500, hist = False)

<matplotlib.axes._subplots.AxesSubplot at 0x1a299f96a0>

Lets' visualize and analyse the date columns¶

[col for col in train_data.columns if train_data[col].dtype == 'datetime64[ns]']

['date_recorded']

1. date_recorded - The date the row was entered¶

train_data['date_recorded'].head()

0   2011-03-14
1   2013-03-06
2   2013-02-25
3   2013-01-28
4   2011-07-13
Name: date_recorded, dtype: datetime64[ns]

year_f = lambda x: x.year

train_data['year_recorded'] = train_data['date_recorded'].apply(year_f)

train_data[['date_recorded','year_recorded']].head()

year_rcrd = train_data['year_recorded'].astype('str')

year_rcrd = year_rcrd.value_counts()

ax = year_rcrd.plot(kind = 'bar' , figsize=(16, 5))
for p in ax.patches:
    ax.annotate(str(p.get_height()), (p.get_x() * 1.007, p.get_height() * 1.007))
plt.xlabel(' Year Recorded ')
plt.grid(True)

train_data['month_recorded'] = train_data['date_recorded'].apply(lambda x: x.month)

train_data[['date_recorded','month_recorded']].head()

month_rcrd = train_data['month_recorded'].astype('str')

ax = month_rcrd.value_counts().plot(kind = 'bar', figsize=(17, 8))
for p in ax.patches:
    ax.annotate(str(p.get_height()), (p.get_x() * 1.007, p.get_height() * 1.007))
plt.xlabel(' Month Recorded ')
plt.grid(True)

*** Insight 11 - FEB and MARCH are the month where most surveys are conducted followed by july , jan and october. Mid year has very few surveys. Perhaps , survey should be distributed equally throughout the year , so that status of pumps are noticed before it's too late .¶

2. construction_year - Year the waterpoint was constructed¶

train_data['construction_year'].isna().sum()

0

train_data[['date_recorded','construction_year','year_recorded']].head()

train_data['years_elapsed'] = train_data['year_recorded'] - train_data['construction_year']

train_data[['date_recorded','construction_year','year_recorded','years_elapsed']].head()

len(train_data[train_data['construction_year'] == 0])

20709

Since a quite a huge number of records are without the year of construction , the intuition here is to replace/impute this with the mode of the fetaure sans zero.¶

Thus , we can engineer a new feature that gives the life of the pumps.¶

from scipy.stats import mode
mode(train_data[train_data['construction_year'] != 0]['construction_year'])

ModeResult(mode=array([2010], dtype=int64), count=array([2645]))

Mode of the construction_year is 2010¶

train_data['const_yr_impt_2010'] = np.where(train_data['construction_year'] == 0, 2010, train_data['construction_year'])

train_data['const_yr_impt_2010'].head()

0    1999
1    2010
2    2009
3    1986
4    2010
Name: const_yr_impt_2010, dtype: int64

train_data['years_elapsed'] = train_data['year_recorded'] - train_data['const_yr_impt_2010']

train_data[['date_recorded','construction_year','const_yr_impt_2010','year_recorded','years_elapsed']].head()

train_data.head(10)

#### 34.93809275	-9.85632177  
#### Lake Nysa - 11.6701° S, 34.6857° E

train_data['basin'].value_counts()

Lake Victoria              10248
Pangani                     8940
Rufiji                      7976
Internal                    7785
Lake Tanganyika             6432
Wami / Ruvu                 5987
Lake Nyasa                  5085
Ruvuma / Southern Coast     4493
Lake Rukwa                  2454
Name: basin, dtype: int64

# function to convert degree, minutes , seconds to just to degree .
def conv_degree(lat):
    lat = lat.replace(" ","")
    deg, minutes, seconds, direction =  re.split('[°\'"]', lat)
    return (float(deg) + float(minutes)/60 + float(seconds)/(60*60)) * (-1 if direction in ['W', 'S'] else 1)

# 'Lake Victoria' = Latitude: -1° 00' 0.00" S Longitude: 33° 00' 0.00" E
lat_1 = conv_degree("""-1° 00' 0.00" S""")
log_1 = conv_degree("""33° 00' 0.00" E""")

# 'Pangani' = Latitude: -5° 25' 30.94" S Longitude: 38° 58' 29.03" E
lat_2 = conv_degree("""-5° 25' 30.94" S""")
log_2 = conv_degree("""38° 58' 29.03" E""")

# 'Rufiji' = Latitude: -8° 00' 0.00" S Longitude: 39° 19' 60.00" E
lat_3 = conv_degree("""-8° 00' 0.00" S""")
log_3 = conv_degree("""39° 19' 60.00" E""")

# 'Internal'(same as ruvu) = Latitude: -6° 48' 17.99" S Longitude: 38° 41' 59.99" E
lat_4 = conv_degree("""-6° 48' 17.99" S""")
log_4 = conv_degree("""38° 41' 59.99" E""")

# 'Lake Tanganyika' = 6.2556° S, 29.5108° E
lat_5 = conv_degree("""6.2556° 0' 00" S""")
log_5 = conv_degree("""29.5108° 0' 00" E""")

# 'Wami / Ruvu' - Latitude: -6° 07' 60.00" S Longitude: 38° 48' 59.99" E
lat_6 = conv_degree("""-6° 07' 60.00" S""")
log_6 = conv_degree("""38° 48' 59.99" E""")

# 'Lake Nyasa' = 11.6701° S, 34.6857° E
lat_7 = conv_degree("""11.6701° 0' 00" S""")
log_7 = conv_degree("""34.6857° 0' 00" E""")

# 'Ruvuma / Southern Coast' = 10.6879° S, 36.2631° E
lat_8 = conv_degree("""10.6879° 0' 00" S""")
log_8 = conv_degree("""36.2631° 0' 00" E""")

#'Lake Rukwa' = 8.0167° S, 32.2654° E
lat_9 = conv_degree("""8.0167° 0' 00" S""")
log_9 = conv_degree("""32.2654° 0' 00" E""")

def get_basin_lat_long(basin , lat_or_long):
    if basin == "Lake Victoria":
        latitude = lat_1
        longitude = log_1
    elif basin == "Pangani":
        latitude = lat_2
        longitude = log_2
    elif basin == "Rufiji":
        latitude = lat_3
        longitude = log_3
    elif basin == "Internal":
        latitude = lat_4
        longitude = log_4
    elif basin == "Lake Tanganyika":
        latitude = lat_5
        longitude = log_5
    elif basin == "Wami / Ruvu":
        latitude = lat_6
        longitude = log_6
    elif basin == "Lake Nyasa":
        latitude = lat_7
        longitude = log_7
    elif basin == "Ruvuma / Southern Coast":
        latitude = lat_8
        longitude = log_8
    elif basin == "Lake Rukwa":
        latitude = lat_9
        longitude = log_9
    
    if lat_or_long.lower() == 'latitude':
        return latitude
    elif lat_or_long.lower() == 'longitude':
        return longitude

get_basin_lat_long("Lake Tanganyika", 'latitude')

-6.2556

train_data['basin_latitude'] = train_data['basin'].apply(get_basin_lat_long, lat_or_long = 'latitude')

train_data['basin_longitude'] = train_data['basin'].apply(get_basin_lat_long, lat_or_long = 'longitude')

# function to calculate the distance of pumps from the respective basins in KMs
def dist_from_basin(basin_lat, basin_long, pump_lat, pump_long):
    coords_1 = (basin_lat.astype('float64'), basin_long.astype('float64'))
    coords_2 = (pump_lat.astype('float64'), pump_long.astype('float64'))
    
    return geopy.distance.vincenty(coords_1, coords_2).km

dist_list = []
for index, row in train_data[['basin_latitude','basin_longitude','latitude','longitude']].iterrows():
    dist = dist_from_basin(row['basin_latitude'], row['basin_longitude'], row['latitude'], row['longitude'])
    dist_list.append(dist)

/anaconda3/lib/python3.6/site-packages/ipykernel_launcher.py:6: DeprecationWarning: Vincenty is deprecated and is going to be removed in geopy 2.0. Use `geopy.distance.geodesic` (or the default `geopy.distance.distance`) instead, which is more accurate and always converges.

len(dist_list)

59400

train_data['basin_pump_dist'] = pd.DataFrame({'basin_pump_dist':dist_list})

train_data[['basin_latitude','basin_longitude','latitude','longitude','basin_pump_dist']].head()

train_data['basin_pump_dist'].isna().sum()

0

Let's visualize the collinearity among the numerical features¶

num_features = [col for col in train_data.columns if train_data[col].dtype in ('int64', 'float64')]

sns.pairplot(train_data[num_features])

<seaborn.axisgrid.PairGrid at 0x1a2609d748>

plt.figure(figsize=(17, 10))
sns.heatmap(train_data[num_features].corr(), annot=True)

<matplotlib.axes._subplots.AxesSubplot at 0x1a32b2dba8>

The above data is very redundant , let's drop unuseful features and then redraw the graph.¶

redundant_cols = ['id','longitude', 'latitude', 'num_private','region_code','construction_year','year_recorded','month_recorded','basin_latitude','basin_longitude']

train_data[num_features].drop(redundant_cols, axis=1).head()

sns.pairplot(train_data[num_features].drop(redundant_cols, axis=1))

<seaborn.axisgrid.PairGrid at 0x1a4349d978>

plt.figure(figsize=(17,10))
sns.heatmap((train_data[num_features].drop(redundant_cols, axis=1)).corr(), annot=True)

<matplotlib.axes._subplots.AxesSubplot at 0x1a496e0d30>

So all these features are important as they are not highly co-related in either direction.¶

Let's now perform some preprocessing to give the data right structure so that it can be modelled using ML algo.¶

train_data_1 = train_data.drop(redundant_cols,axis=1)

train_data_1.shape

(59400, 37)

train_data_1.isna().sum()

amount_tsh                   0
date_recorded                0
funder                    3635
gps_height                   0
installer                 3655
wpt_name                     0
basin                        0
subvillage                 371
region                       0
district_code                0
lga                          0
ward                         0
population                   0
public_meeting            3334
recorded_by                  0
scheme_management         3877
scheme_name              28166
permit                    3056
extraction_type              0
extraction_type_group        0
extraction_type_class        0
management                   0
management_group             0
payment                      0
payment_type                 0
water_quality                0
quality_group                0
quantity                     0
quantity_group               0
source                       0
source_type                  0
source_class                 0
waterpoint_type              0
waterpoint_type_group        0
years_elapsed                0
const_yr_impt_2010           0
basin_pump_dist              0
dtype: int64

del(train_data)

imputing funder with text 'funder_missing'¶

train_data_1['funder'].fillna('funder_missing', inplace=True)

imputing installer with text 'installer_missing'¶

train_data_1['installer'].fillna('installer_missing', inplace=True)

We choose to ignore sub-village column for modelling as we are other predictors for location¶

train_data_1.drop('subvillage', axis=1, inplace=True)

Impute the public_meeting with text 'miss_p_mtng'¶

train_data_1['public_meeting'].fillna('miss_p_mtng', inplace=True)

For scheme_management(3877) and scheme_name(28166) , we can drop scheme_name as scheme_name can be identified with scheme_management , will impute scheme_management with text 'miss_sch_mngt'.¶

train_data_1['scheme_management'].fillna('miss_sch_mngt', inplace=True)

train_data_1.drop('scheme_name', axis=1, inplace=True)

For Permit we impute with text 'miss_permit', it doesn't mean 'NO' permit though.¶

train_data_1['permit'].fillna('miss_permit', inplace=True)

##### Now again check if there is any missing or null values
train_data_1[train_data_1.isna().any(axis=1)]

train_data_1.shape

(59400, 35)

dropping the date_recorded and const_yr_impt_2010 as the info from these 2 are incorporated in years_elpased¶

train_data_1.drop(['date_recorded','const_yr_impt_2010'], inplace=True, axis=1)

train_data_1.columns

Index(['amount_tsh', 'funder', 'gps_height', 'installer', 'wpt_name', 'basin',
       'region', 'district_code', 'lga', 'ward', 'population',
       'public_meeting', 'recorded_by', 'scheme_management', 'permit',
       'extraction_type', 'extraction_type_group', 'extraction_type_class',
       'management', 'management_group', 'payment', 'payment_type',
       'water_quality', 'quality_group', 'quantity', 'quantity_group',
       'source', 'source_type', 'source_class', 'waterpoint_type',
       'waterpoint_type_group', 'years_elapsed', 'basin_pump_dist'],
      dtype='object')

From the above analysis of the variables, it has been discovered that there are many features which are redundant , which will cause below issue :¶

1. Delay in training.¶

2. High utilization of cumputing resources.¶

3. Over-fiiting/high-variance¶

4. Difficulty in interpreting the model.¶

hence it is better to drop these features.¶

redundant_cols_new = ['district_code',
'waterpoint_type_group',
'source_type',
'source_class',
'quantity_group',
'quality_group',
'payment_type',
'management_group',
'extraction_type_group',
'extraction_type_class',
'recorded_by',
'lga',
'ward',
'basin',
'wpt_name']

train_data_1.drop(redundant_cols_new, axis=1, inplace=True)

train_data_1.columns

Index(['amount_tsh', 'funder', 'gps_height', 'installer', 'region',
       'population', 'public_meeting', 'scheme_management', 'permit',
       'extraction_type', 'management', 'payment', 'water_quality', 'quantity',
       'source', 'waterpoint_type', 'years_elapsed', 'basin_pump_dist'],
      dtype='object')

train_data_1.shape

(59400, 18)

train_data_1.head()

# get the left over numerical columns for scaling
num_col_sc = [col for col in train_data_1.columns if train_data_1[col].dtype in('int64','float64')]

num_col_sc

['amount_tsh', 'gps_height', 'population', 'years_elapsed', 'basin_pump_dist']

train_data_1 = pd.get_dummies(train_data_1)

train_data_1.shape

(59400, 4156)

train_data_1.head()

Perform the scaling of the features¶

from sklearn.preprocessing import StandardScaler

sc = StandardScaler()

train_data_1[num_col_sc] = sc.fit_transform(train_data_1[num_col_sc])

/anaconda3/lib/python3.6/site-packages/sklearn/preprocessing/data.py:645: DataConversionWarning: Data with input dtype int64, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
/anaconda3/lib/python3.6/site-packages/sklearn/base.py:464: DataConversionWarning: Data with input dtype int64, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, **fit_params).transform(X)

train_data_1.head()

col_lst = train_data_1.columns

col_lst

Index(['amount_tsh', 'gps_height', 'population', 'years_elapsed',
       'basin_pump_dist', 'funder_0', 'funder_A/co Germany', 'funder_Aar',
       'funder_Abas Ka', 'funder_Abasia',
       ...
       'source_shallow well', 'source_spring', 'source_unknown',
       'waterpoint_type_cattle trough', 'waterpoint_type_communal standpipe',
       'waterpoint_type_communal standpipe multiple', 'waterpoint_type_dam',
       'waterpoint_type_hand pump', 'waterpoint_type_improved spring',
       'waterpoint_type_other'],
      dtype='object', length=4156)

# dropping one column each from the group of newly created dummy columns to ward off the issue of multi-collinearity
train_data_1.drop(['funder_funder_missing','installer_installer_missing','region_Iringa',
                   'public_meeting_miss_p_mtng','scheme_management_miss_sch_mngt','permit_miss_permit',
                  'management_other - school','payment_unknown','water_quality_unknown',
                  'quantity_unknown','source_unknown','water_quality_unknown'], axis=1, inplace=True)

train_data_1.shape

(59400, 4145)

Import the training target data-set¶

# import the train-target dataset
train_target = pd.read_csv('raw_train_target_vectors.csv')

train_target.shape

(59400, 2)

train_target.head()

train_target['status_group'].isna().sum()

0

train_target['status_group'].unique()

array(['functional', 'non functional', 'functional needs repair'],
      dtype=object)

pre-processing the target variable from being categorical to numerical¶

train_target['status_group'] = train_target['status_group'].map({'functional':0,
                                                                 'non functional':1,
                                                                 'functional needs repair':2})

train_target.head()

0    0
1    0
2    0
3    1
4    0
Name: status_group, dtype: int64

train_target = train_target['status_group']

Now let's segregate the entire data set into train , validation and test sets¶

from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(train_data_1,train_target,test_size=.15,
                                                    random_state=321)

## get validation set from train set
X_train, X_valid, y_train, y_valid = train_test_split(X_train, y_train, test_size=.2, 
                                                      random_state=313)

X_train.shape, X_valid.shape, X_test.shape, y_train.shape, y_valid.shape, y_test.shape

((40392, 4145), (10098, 4145), (8910, 4145), (40392,), (10098,), (8910,))

At this moment I am compelled to perform dimensionality reduction step - namely PCA to compress my feature space of several thousand features.¶

from sklearn.decomposition import PCA

pca = PCA(.95)
pca.fit(X_train)

#pca_samples = pca.transform(train_data_1)

PCA(copy=True, iterated_power='auto', n_components=0.95, random_state=None,
  svd_solver='auto', tol=0.0, whiten=False)

pca.n_components_

136

pca.explained_variance_ratio_

array([0.12822833, 0.0892335 , 0.08197641, 0.07738054, 0.05933666,
       0.04592652, 0.03761974, 0.03254067, 0.0292746 , 0.02666826,
       0.02030454, 0.0169611 , 0.01622157, 0.01521431, 0.01304298,
       0.01219111, 0.011536  , 0.01099073, 0.0098409 , 0.00953179,
       0.00884177, 0.00763352, 0.00735443, 0.00729742, 0.00657773,
       0.00595982, 0.0057316 , 0.00516616, 0.00498214, 0.00491715,
       0.00451783, 0.00436116, 0.00418003, 0.00390396, 0.00381815,
       0.00374392, 0.00365451, 0.00358327, 0.00350678, 0.00329978,
       0.00309766, 0.00299255, 0.0029528 , 0.00282128, 0.00278156,
       0.00265938, 0.00261969, 0.00258057, 0.00243823, 0.0023699 ,
       0.00224707, 0.00218801, 0.00215508, 0.00204343, 0.0020147 ,
       0.00194403, 0.00188566, 0.0018381 , 0.00178667, 0.0016899 ,
       0.00163151, 0.00160954, 0.00158515, 0.00154052, 0.00146876,
       0.00143221, 0.00141691, 0.00138489, 0.00134239, 0.00131341,
       0.00129063, 0.00126102, 0.00116261, 0.00112661, 0.00111006,
       0.0010919 , 0.00107895, 0.00101405, 0.00100175, 0.00096405,
       0.00094858, 0.00089288, 0.00086306, 0.00084718, 0.00080482,
       0.0007981 , 0.0007848 , 0.000781  , 0.00075451, 0.00075242,
       0.00072239, 0.00069709, 0.00068323, 0.00067332, 0.00065102,
       0.00064152, 0.00062645, 0.00059968, 0.00059874, 0.00057203,
       0.00056631, 0.00055281, 0.00053592, 0.00053434, 0.00051778,
       0.00051372, 0.0005058 , 0.00048885, 0.00048145, 0.00047797,
       0.00047221, 0.00046802, 0.00046309, 0.00045528, 0.00043991,
       0.00043818, 0.00043685, 0.00042353, 0.00041998, 0.00041386,
       0.00040765, 0.00040508, 0.00040124, 0.00039463, 0.00039142,
       0.00038606, 0.000385  , 0.00038283, 0.00037702, 0.00037091,
       0.00036269, 0.00035417, 0.00034932, 0.00034282, 0.00033756,
       0.00033566])

for n_compnt in [50,60,70,75,80,85,90,95,100,110,120,125,130,135]:
    print('{} % of variance is explained by first {} components.'
      .format((round(np.cumsum(pca.explained_variance_ratio_)[n_compnt],2) * 100),n_compnt))

88.0 % of variance is explained by first 50 components.
90.0 % of variance is explained by first 60 components.
91.0 % of variance is explained by first 70 components.
92.0 % of variance is explained by first 75 components.
92.0 % of variance is explained by first 80 components.
93.0 % of variance is explained by first 85 components.
93.0 % of variance is explained by first 90 components.
93.0 % of variance is explained by first 95 components.
94.0 % of variance is explained by first 100 components.
94.0 % of variance is explained by first 110 components.
94.0 % of variance is explained by first 120 components.
95.0 % of variance is explained by first 125 components.
95.0 % of variance is explained by first 130 components.
95.0 % of variance is explained by first 135 components.

Apply the mapping/transformation to train, validation and test sets.¶

X_train = pca.transform(X_train)
X_valid = pca.transform(X_valid)
X_test = pca.transform(X_test)

X_train.shape , X_valid.shape, X_test.shape

((40392, 136), (10098, 136), (8910, 136))

X_train[:5,:5]

array([[ 0.09558042,  0.20629677,  1.10368332, -1.43542895, -1.27141352],
       [-1.56349914, -0.07807022,  0.06527295,  0.92176458, -0.39106082],
       [ 0.8930848 , -0.27636336,  0.90167219, -0.51384158,  0.07970041],
       [ 0.9443107 , -0.06926744, -0.70016221, -0.05104971,  1.31398127],
       [-0.72705389, -0.12687915, -0.72443054,  1.20887384,  0.58719472]])

Model Selection steps¶

from sklearn.metrics import accuracy_score
from time import time

def probable_models(classifier, train_ftr_set, valid_ftr_set, train_trgt_set, valid_trgt_set):
    '''
    defining a function to iterative run over a list of classifiers and generate a 
    comparison matrix
    '''
    
    result = {}
    
    start_tm = time()
    classifier = classifier.fit(train_ftr_set, train_trgt_set)
    end_tm = time()
    
    result['train_score'] = classifier.score(train_ftr_set, train_trgt_set)
    result['valid_score'] = classifier.score(valid_ftr_set, valid_trgt_set)
    result['time'] = end_tm - start_tm
    
    start_tm = 0
    end_tm = 0
    
    return result

from sklearn.svm import SVC
from sklearn.naive_bayes import GaussianNB
from sklearn.neighbors import KNeighborsClassifier
from sklearn.tree import DecisionTreeClassifier
from sklearn.ensemble import RandomForestClassifier, AdaBoostClassifier, GradientBoostingClassifier, ExtraTreesClassifier
from sklearn.neural_network import MLPClassifier

clf_a = SVC(random_state = 313)
clf_b = GaussianNB()
clf_c = KNeighborsClassifier()
clf_d = DecisionTreeClassifier(random_state = 313)
clf_e = RandomForestClassifier(criterion='entropy', random_state = 313)
clf_f = AdaBoostClassifier(random_state = 313)
clf_g = GradientBoostingClassifier(random_state = 313)
clf_h = ExtraTreesClassifier(criterion='entropy', random_state=313)
clf_i = MLPClassifier(random_state = 313)

results = {}

for clf in [clf_a, clf_b, clf_c, clf_d, clf_e, clf_f, clf_g, clf_h, clf_i]:
    name = clf.__class__.__name__
    results[name] = probable_models(clf, X_train, X_valid, y_train, y_valid)

/anaconda3/lib/python3.6/site-packages/sklearn/svm/base.py:196: FutureWarning: The default value of gamma will change from 'auto' to 'scale' in version 0.22 to account better for unscaled features. Set gamma explicitly to 'auto' or 'scale' to avoid this warning.
  "avoid this warning.", FutureWarning)
/anaconda3/lib/python3.6/site-packages/sklearn/ensemble/forest.py:246: FutureWarning: The default value of n_estimators will change from 10 in version 0.20 to 100 in 0.22.
  "10 in version 0.20 to 100 in 0.22.", FutureWarning)
/anaconda3/lib/python3.6/site-packages/sklearn/ensemble/forest.py:246: FutureWarning: The default value of n_estimators will change from 10 in version 0.20 to 100 in 0.22.
  "10 in version 0.20 to 100 in 0.22.", FutureWarning)
/anaconda3/lib/python3.6/site-packages/sklearn/neural_network/multilayer_perceptron.py:562: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (200) reached and the optimization hasn't converged yet.
  % self.max_iter, ConvergenceWarning)

results

{'SVC': {'train_score': 0.7376213111507229,
  'valid_score': 0.7370766488413547,
  'accuracy': 0.7370766488413547,
  'time': 228.85598397254944},
 'GaussianNB': {'train_score': 0.4734353337294514,
  'valid_score': 0.4726678550207962,
  'accuracy': 0.4726678550207962,
  'time': 0.10456705093383789},
 'KNeighborsClassifier': {'train_score': 0.8353881956823134,
  'valid_score': 0.7756981580510992,
  'accuracy': 0.7756981580510992,
  'time': 0.23358511924743652},
 'DecisionTreeClassifier': {'train_score': 0.9974499900970489,
  'valid_score': 0.7265795206971678,
  'accuracy': 0.7265795206971678,
  'time': 11.238741874694824},
 'RandomForestClassifier': {'train_score': 0.9784363240245593,
  'valid_score': 0.7687660922955041,
  'accuracy': 0.7687660922955041,
  'time': 6.3978822231292725},
 'AdaBoostClassifier': {'train_score': 0.7049168152109329,
  'valid_score': 0.7021192315309962,
  'accuracy': 0.7021192315309962,
  'time': 31.82717800140381},
 'GradientBoostingClassifier': {'train_score': 0.7704991087344029,
  'valid_score': 0.7625272331154684,
  'accuracy': 0.7625272331154684,
  'time': 168.3458070755005},
 'ExtraTreesClassifier': {'train_score': 0.9974499900970489,
  'valid_score': 0.7624282036046742,
  'accuracy': 0.7624282036046742,
  'time': 1.077476978302002},
 'MLPClassifier': {'train_score': 0.8657902554961379,
  'valid_score': 0.773717567835215,
  'accuracy': 0.773717567835215,
  'time': 35.131067991256714}}

model_selection_df = pd.DataFrame(results).T

model_selection_df

t1 = time()

t2 = time()

t2 - t1

7.368470191955566

model_selection_df.T.plot(kind = 'bar', figsize=(17,10))

<matplotlib.axes._subplots.AxesSubplot at 0x1a8300e198>

Difficult to visualize the comparative graph, so let's normalize the data and see.¶

model_selection_df['time'] = (model_selection_df['time'] - min(model_selection_df['time']))/(max(model_selection_df['time']) - min(model_selection_df['time']))
model_selection_df['train_score'] = (model_selection_df['train_score'] - min(model_selection_df['train_score']))/(max(model_selection_df['train_score']) - min(model_selection_df['train_score']))
model_selection_df['valid_score'] = (model_selection_df['valid_score'] - min(model_selection_df['valid_score']))/(max(model_selection_df['valid_score']) - min(model_selection_df['valid_score']))

model_selection_df

model_selection_df.T.plot(kind='bar', figsize=(17,10))

<matplotlib.axes._subplots.AxesSubplot at 0x1a829b6f60>

Hence, from above it's clear that we will try out with KNN and MLP for further parameter tuning and attain better results.¶

Let's now perform the grid-serach with cross-validation to find the best hyper-parameters for KNN.¶

from sklearn.model_selection import GridSearchCV
from sklearn.metrics import make_scorer, accuracy_score

# let's define the parameters dictionary for KNN
'''
parameters = {'n_neighbors' :list(range(20,150,10)),
              'algorithm' : ['auto', 'ball_tree', 'kd_tree', 'brute'],
              'leaf_size' : list(range(20,150,10)),
              'p':[1,2, 3, 4]
             }
'''
parameters = {'n_neighbors' :[5],
              'algorithm' : ['auto', 'ball_tree', 'kd_tree', 'brute'],
              'leaf_size' : [30],
              'p':[2]
             }

start_tm = time()

scorer = make_scorer(accuracy_score)
clf = KNeighborsClassifier()
grid_obj = GridSearchCV(clf,
                        param_grid=parameters,
                        scoring=scorer,
                        ##cv=10,
                        n_jobs=-1)
grid_fit = grid_obj.fit(X_train, y_train)
best_clf = grid_fit.best_estimator_
base_pred = (clf.fit(X_train, y_train)).predict(X_valid)
best_pred = best_clf.predict(X_valid)

end_tm = time()

print('The prediction by base KNN validation set is : {}'.format(accuracy_score(y_valid, base_pred)),
      '\nThe prediction by optimized KNN on validation set is : {}'.format(accuracy_score(y_valid, best_pred)),
      '\nTime taken is : {:.4f} sec'.format(end_tm - start_tm))

/anaconda3/lib/python3.6/site-packages/sklearn/model_selection/_split.py:2053: FutureWarning: You should specify a value for 'cv' instead of relying on the default value. The default value will change from 3 to 5 in version 0.22.
  warnings.warn(CV_WARNING, FutureWarning)

The prediction by base KNN validation set is : 0.7756981580510992 
The prediction by optimized KNN on validation set is : 0.7753020400079224 
Time taken is : 409.0240 sec

Capturing the run of individual combinations as complete grid was taking too much of time. Due to time constarint , as I have to submit the analysis for Axiata Analytics.¶

best_clf.get_params

<bound method BaseEstimator.get_params of KNeighborsClassifier(algorithm='brute', leaf_size=30, metric='minkowski',
           metric_params=None, n_jobs=None, n_neighbors=5, p=2,
           weights='uniform')>

KNeighborsClassifier(algorithm='kd_tree', leaf_size=30, metric='minkowski',
           metric_params=None, n_jobs=None, n_neighbors=10, p=2,
           weights='uniform')

The prediction by base KNN validation set is : 0.7756981580510992 
The prediction by optimized KNN on validation set is : 0.7712418300653595 
Time taken is : 171.2059 sec

KNeighborsClassifier(algorithm='auto', leaf_size=40, metric='minkowski',
           metric_params=None, n_jobs=None, n_neighbors=10, p=2,
           weights='uniform')
                     
The prediction by base KNN validation set is : 0.7756981580510992 
The prediction by optimized KNN on validation set is : 0.7713408595761537 
Time taken is : 151.4696 sec

KNeighborsClassifier(algorithm='auto', leaf_size=30, metric='minkowski',
           metric_params=None, n_jobs=None, n_neighbors=15, p=2,
           weights='uniform')

The prediction by base KNN validation set is : 0.7756981580510992 
The prediction by optimized KNN on validation set is : 0.767379679144385 
Time taken is : 181.1374 sec

KNeighborsClassifier(algorithm='kd_tree', leaf_size=30, metric='minkowski',
           metric_params=None, n_jobs=None, n_neighbors=5, p=2,
           weights='uniform')

The prediction by base KNN validation set is : 0.7756981580510992 
The prediction by optimized KNN on validation set is : 0.7756981580510992 
Time taken is : 109.9834 sec

'algorithm' : ['auto', 'ball_tree', 'kd_tree', 'brute'],
KNeighborsClassifier(algorithm='brute', leaf_size=30, metric='minkowski',
           metric_params=None, n_jobs=None, n_neighbors=5, p=2,
           weights='uniform')

The prediction by base KNN validation set is : 0.7756981580510992 
The prediction by optimized KNN on validation set is : 0.7753020400079224 
Time taken is : 409.0240 sec

Now trying out with Neural Networks using keras with tensorflow backend¶

from keras.models import Sequential
from keras.layers import Dense, Activation,Dropout
from keras.optimizers import Adam,SGD,RMSprop
from keras import regularizers
from keras.callbacks import ModelCheckpoint
from sklearn.metrics import accuracy_score
from keras.utils import np_utils

X_train.shape

(40392, 136)

nb_classes = 3
y_train = np_utils.to_categorical(y_train, nb_classes)
y_valid = np_utils.to_categorical(y_valid, nb_classes)
y_test = np_utils.to_categorical(y_test, nb_classes)

y_train.shape, y_valid.shape, y_test.shape

((40392, 3), (10098, 3), (8910, 3))

# first model architechture
np.random.seed(13)

model = Sequential()
model.add(Dense(1024, kernel_regularizer=regularizers.l2(.001),input_dim=X_train.shape[1]))
model.add(Activation('relu'))
model.add(Dropout(0.2))
model.add(Dense(512,kernel_regularizer=regularizers.l2(.001)))
model.add(Activation('relu'))
model.add(Dropout(0.2))
model.add(Dense(128,kernel_regularizer=regularizers.l1_l2(.001)))
model.add(Activation('relu'))
model.add(Dropout(0.3))
model.add(Dense(32, kernel_regularizer=regularizers.l2(.001)))
model.add(Activation('relu'))
model.add(Dense(3, activation = 'softmax'))

model.summary()

_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
dense_42 (Dense)             (None, 1024)              140288    
_________________________________________________________________
activation_33 (Activation)   (None, 1024)              0         
_________________________________________________________________
dropout_25 (Dropout)         (None, 1024)              0         
_________________________________________________________________
dense_43 (Dense)             (None, 512)               524800    
_________________________________________________________________
activation_34 (Activation)   (None, 512)               0         
_________________________________________________________________
dropout_26 (Dropout)         (None, 512)               0         
_________________________________________________________________
dense_44 (Dense)             (None, 128)               65664     
_________________________________________________________________
activation_35 (Activation)   (None, 128)               0         
_________________________________________________________________
dropout_27 (Dropout)         (None, 128)               0         
_________________________________________________________________
dense_45 (Dense)             (None, 32)                4128      
_________________________________________________________________
activation_36 (Activation)   (None, 32)                0         
_________________________________________________________________
dense_46 (Dense)             (None, 3)                 99        
=================================================================
Total params: 734,979
Trainable params: 734,979
Non-trainable params: 0
_________________________________________________________________

# compile the model
model.compile(optimizer='rmsprop', loss='categorical_crossentropy', metrics=['accuracy'])

from keras.callbacks import ModelCheckpoint  

# checkpoint
filepath="weights-improvement-best-model.hdf5"
checkpoint = ModelCheckpoint(filepath, monitor='val_loss', verbose=1, save_best_only=True, mode='min')
callbacks_list_1 = [checkpoint]

# training the model

start_time = time()
model.fit(X_train,y_train,batch_size= 10000, epochs= 500,
          validation_data=(X_valid,y_valid),
          #validation_split = 0.2,
         verbose =2,callbacks = callbacks_list_1)
end_time = time()

Train on 40392 samples, validate on 10098 samples
Epoch 1/500
 - 3s - loss: 6.6574 - acc: 0.4958 - val_loss: 5.6679 - val_acc: 0.6596

Epoch 00001: val_loss improved from inf to 5.66787, saving model to weights-improvement-best-model.hdf5
Epoch 2/500
 - 3s - loss: 5.4453 - acc: 0.6364 - val_loss: 4.8705 - val_acc: 0.7239

Epoch 00002: val_loss improved from 5.66787 to 4.87046, saving model to weights-improvement-best-model.hdf5
Epoch 3/500
 - 3s - loss: 4.6836 - acc: 0.7190 - val_loss: 4.3321 - val_acc: 0.6666

Epoch 00003: val_loss improved from 4.87046 to 4.33205, saving model to weights-improvement-best-model.hdf5
Epoch 4/500
 - 3s - loss: 4.1889 - acc: 0.7001 - val_loss: 3.7808 - val_acc: 0.7408

Epoch 00004: val_loss improved from 4.33205 to 3.78081, saving model to weights-improvement-best-model.hdf5
Epoch 5/500
 - 3s - loss: 3.6601 - acc: 0.7374 - val_loss: 3.3519 - val_acc: 0.7490

Epoch 00005: val_loss improved from 3.78081 to 3.35190, saving model to weights-improvement-best-model.hdf5
Epoch 6/500
 - 3s - loss: 3.2631 - acc: 0.7323 - val_loss: 2.9859 - val_acc: 0.7444

Epoch 00006: val_loss improved from 3.35190 to 2.98592, saving model to weights-improvement-best-model.hdf5
Epoch 7/500
 - 3s - loss: 2.8870 - acc: 0.7387 - val_loss: 2.6629 - val_acc: 0.7411

Epoch 00007: val_loss improved from 2.98592 to 2.66293, saving model to weights-improvement-best-model.hdf5
Epoch 8/500
 - 3s - loss: 2.5741 - acc: 0.7358 - val_loss: 2.3390 - val_acc: 0.7483

Epoch 00008: val_loss improved from 2.66293 to 2.33904, saving model to weights-improvement-best-model.hdf5
Epoch 9/500
 - 3s - loss: 2.2601 - acc: 0.7476 - val_loss: 2.0643 - val_acc: 0.7495

Epoch 00009: val_loss improved from 2.33904 to 2.06429, saving model to weights-improvement-best-model.hdf5
Epoch 10/500
 - 3s - loss: 2.0135 - acc: 0.7413 - val_loss: 1.8560 - val_acc: 0.7372

Epoch 00010: val_loss improved from 2.06429 to 1.85595, saving model to weights-improvement-best-model.hdf5
Epoch 11/500
 - 3s - loss: 1.7859 - acc: 0.7457 - val_loss: 1.6394 - val_acc: 0.7424

Epoch 00011: val_loss improved from 1.85595 to 1.63938, saving model to weights-improvement-best-model.hdf5
Epoch 12/500
 - 3s - loss: 1.5972 - acc: 0.7398 - val_loss: 1.4530 - val_acc: 0.7535

Epoch 00012: val_loss improved from 1.63938 to 1.45298, saving model to weights-improvement-best-model.hdf5
Epoch 13/500
 - 3s - loss: 1.4161 - acc: 0.7474 - val_loss: 1.3495 - val_acc: 0.7273

Epoch 00013: val_loss improved from 1.45298 to 1.34951, saving model to weights-improvement-best-model.hdf5
Epoch 14/500
 - 3s - loss: 1.2843 - acc: 0.7425 - val_loss: 1.1681 - val_acc: 0.7566

Epoch 00014: val_loss improved from 1.34951 to 1.16806, saving model to weights-improvement-best-model.hdf5
Epoch 15/500
 - 3s - loss: 1.1543 - acc: 0.7460 - val_loss: 1.1005 - val_acc: 0.7500

Epoch 00015: val_loss improved from 1.16806 to 1.10054, saving model to weights-improvement-best-model.hdf5
Epoch 16/500
 - 3s - loss: 1.0597 - acc: 0.7500 - val_loss: 1.0057 - val_acc: 0.7459

Epoch 00016: val_loss improved from 1.10054 to 1.00575, saving model to weights-improvement-best-model.hdf5
Epoch 17/500
 - 3s - loss: 0.9922 - acc: 0.7433 - val_loss: 0.9442 - val_acc: 0.7480

Epoch 00017: val_loss improved from 1.00575 to 0.94418, saving model to weights-improvement-best-model.hdf5
Epoch 18/500
 - 3s - loss: 0.9300 - acc: 0.7468 - val_loss: 0.8732 - val_acc: 0.7602

Epoch 00018: val_loss improved from 0.94418 to 0.87323, saving model to weights-improvement-best-model.hdf5
Epoch 19/500
 - 3s - loss: 0.8733 - acc: 0.7547 - val_loss: 0.8418 - val_acc: 0.7569

Epoch 00019: val_loss improved from 0.87323 to 0.84178, saving model to weights-improvement-best-model.hdf5
Epoch 20/500
 - 3s - loss: 0.8609 - acc: 0.7461 - val_loss: 0.8543 - val_acc: 0.7436

Epoch 00020: val_loss did not improve from 0.84178
Epoch 21/500
 - 3s - loss: 0.8396 - acc: 0.7518 - val_loss: 0.8162 - val_acc: 0.7562

Epoch 00021: val_loss improved from 0.84178 to 0.81622, saving model to weights-improvement-best-model.hdf5
Epoch 22/500
 - 3s - loss: 0.8224 - acc: 0.7516 - val_loss: 0.8027 - val_acc: 0.7585

Epoch 00022: val_loss improved from 0.81622 to 0.80274, saving model to weights-improvement-best-model.hdf5
Epoch 23/500
 - 3s - loss: 0.8131 - acc: 0.7523 - val_loss: 0.7937 - val_acc: 0.7590

Epoch 00023: val_loss improved from 0.80274 to 0.79369, saving model to weights-improvement-best-model.hdf5
Epoch 24/500
 - 3s - loss: 0.7932 - acc: 0.7559 - val_loss: 0.8188 - val_acc: 0.7354

Epoch 00024: val_loss did not improve from 0.79369
Epoch 25/500
 - 3s - loss: 0.8081 - acc: 0.7449 - val_loss: 0.7729 - val_acc: 0.7623

Epoch 00025: val_loss improved from 0.79369 to 0.77289, saving model to weights-improvement-best-model.hdf5
Epoch 26/500
 - 3s - loss: 0.7819 - acc: 0.7557 - val_loss: 0.7881 - val_acc: 0.7508

Epoch 00026: val_loss did not improve from 0.77289
Epoch 27/500
 - 3s - loss: 0.7836 - acc: 0.7515 - val_loss: 0.7656 - val_acc: 0.7580

Epoch 00027: val_loss improved from 0.77289 to 0.76560, saving model to weights-improvement-best-model.hdf5
Epoch 28/500
 - 3s - loss: 0.7769 - acc: 0.7535 - val_loss: 0.7625 - val_acc: 0.7604

Epoch 00028: val_loss improved from 0.76560 to 0.76251, saving model to weights-improvement-best-model.hdf5
Epoch 29/500
 - 3s - loss: 0.7651 - acc: 0.7575 - val_loss: 0.7527 - val_acc: 0.7612

Epoch 00029: val_loss improved from 0.76251 to 0.75272, saving model to weights-improvement-best-model.hdf5
Epoch 30/500
 - 3s - loss: 0.7598 - acc: 0.7575 - val_loss: 0.7561 - val_acc: 0.7574

Epoch 00030: val_loss did not improve from 0.75272
Epoch 31/500
 - 3s - loss: 0.7617 - acc: 0.7561 - val_loss: 0.7713 - val_acc: 0.7491

Epoch 00031: val_loss did not improve from 0.75272
Epoch 32/500
 - 3s - loss: 0.7586 - acc: 0.7558 - val_loss: 0.7516 - val_acc: 0.7549

Epoch 00032: val_loss improved from 0.75272 to 0.75158, saving model to weights-improvement-best-model.hdf5
Epoch 33/500
 - 3s - loss: 0.7562 - acc: 0.7537 - val_loss: 0.7378 - val_acc: 0.7626

Epoch 00033: val_loss improved from 0.75158 to 0.73781, saving model to weights-improvement-best-model.hdf5
Epoch 34/500
 - 3s - loss: 0.7405 - acc: 0.7623 - val_loss: 0.7502 - val_acc: 0.7514

Epoch 00034: val_loss did not improve from 0.73781
Epoch 35/500
 - 3s - loss: 0.7572 - acc: 0.7518 - val_loss: 0.7467 - val_acc: 0.7549

Epoch 00035: val_loss did not improve from 0.73781
Epoch 36/500
 - 3s - loss: 0.7422 - acc: 0.7574 - val_loss: 0.7278 - val_acc: 0.7643

Epoch 00036: val_loss improved from 0.73781 to 0.72777, saving model to weights-improvement-best-model.hdf5
Epoch 37/500
 - 3s - loss: 0.7360 - acc: 0.7604 - val_loss: 0.7287 - val_acc: 0.7656

Epoch 00037: val_loss did not improve from 0.72777
Epoch 38/500
 - 3s - loss: 0.7403 - acc: 0.7559 - val_loss: 0.7483 - val_acc: 0.7540

Epoch 00038: val_loss did not improve from 0.72777
Epoch 39/500
 - 3s - loss: 0.7394 - acc: 0.7574 - val_loss: 0.7192 - val_acc: 0.7671

Epoch 00039: val_loss improved from 0.72777 to 0.71924, saving model to weights-improvement-best-model.hdf5
Epoch 40/500
 - 3s - loss: 0.7258 - acc: 0.7639 - val_loss: 0.7197 - val_acc: 0.7661

Epoch 00040: val_loss did not improve from 0.71924
Epoch 41/500
 - 3s - loss: 0.7228 - acc: 0.7638 - val_loss: 0.7359 - val_acc: 0.7582

Epoch 00041: val_loss did not improve from 0.71924
Epoch 42/500
 - 3s - loss: 0.7417 - acc: 0.7508 - val_loss: 0.7197 - val_acc: 0.7644

Epoch 00042: val_loss did not improve from 0.71924
Epoch 43/500
 - 3s - loss: 0.7216 - acc: 0.7629 - val_loss: 0.7261 - val_acc: 0.7610

Epoch 00043: val_loss did not improve from 0.71924
Epoch 44/500
 - 3s - loss: 0.7321 - acc: 0.7571 - val_loss: 0.7144 - val_acc: 0.7654

Epoch 00044: val_loss improved from 0.71924 to 0.71436, saving model to weights-improvement-best-model.hdf5
Epoch 45/500
 - 3s - loss: 0.7186 - acc: 0.7620 - val_loss: 0.7169 - val_acc: 0.7623

Epoch 00045: val_loss did not improve from 0.71436
Epoch 46/500
 - 3s - loss: 0.7146 - acc: 0.7663 - val_loss: 0.7409 - val_acc: 0.7549

Epoch 00046: val_loss did not improve from 0.71436
Epoch 47/500
 - 3s - loss: 0.7354 - acc: 0.7546 - val_loss: 0.7122 - val_acc: 0.7658

Epoch 00047: val_loss improved from 0.71436 to 0.71222, saving model to weights-improvement-best-model.hdf5
Epoch 48/500
 - 3s - loss: 0.7100 - acc: 0.7670 - val_loss: 0.7059 - val_acc: 0.7673

Epoch 00048: val_loss improved from 0.71222 to 0.70593, saving model to weights-improvement-best-model.hdf5
Epoch 49/500
 - 3s - loss: 0.7074 - acc: 0.7677 - val_loss: 0.7077 - val_acc: 0.7667

Epoch 00049: val_loss did not improve from 0.70593
Epoch 50/500
 - 3s - loss: 0.7340 - acc: 0.7513 - val_loss: 0.7132 - val_acc: 0.7638

Epoch 00050: val_loss did not improve from 0.70593
Epoch 51/500
 - 3s - loss: 0.7072 - acc: 0.7645 - val_loss: 0.7063 - val_acc: 0.7684

Epoch 00051: val_loss did not improve from 0.70593
Epoch 52/500
 - 3s - loss: 0.7034 - acc: 0.7674 - val_loss: 0.7070 - val_acc: 0.7678

Epoch 00052: val_loss did not improve from 0.70593
Epoch 53/500
 - 3s - loss: 0.7190 - acc: 0.7561 - val_loss: 0.7100 - val_acc: 0.7617

Epoch 00053: val_loss did not improve from 0.70593
Epoch 54/500
 - 3s - loss: 0.7038 - acc: 0.7668 - val_loss: 0.7050 - val_acc: 0.7644

Epoch 00054: val_loss improved from 0.70593 to 0.70497, saving model to weights-improvement-best-model.hdf5
Epoch 55/500
 - 3s - loss: 0.7101 - acc: 0.7632 - val_loss: 0.7032 - val_acc: 0.7681

Epoch 00055: val_loss improved from 0.70497 to 0.70317, saving model to weights-improvement-best-model.hdf5
Epoch 56/500
 - 3s - loss: 0.7024 - acc: 0.7671 - val_loss: 0.7085 - val_acc: 0.7606

Epoch 00056: val_loss did not improve from 0.70317
Epoch 57/500
 - 3s - loss: 0.7047 - acc: 0.7660 - val_loss: 0.7089 - val_acc: 0.7636

Epoch 00057: val_loss did not improve from 0.70317
Epoch 58/500
 - 3s - loss: 0.7013 - acc: 0.7673 - val_loss: 0.7550 - val_acc: 0.7251

Epoch 00058: val_loss did not improve from 0.70317
Epoch 59/500
 - 3s - loss: 0.7196 - acc: 0.7563 - val_loss: 0.7014 - val_acc: 0.7687

Epoch 00059: val_loss improved from 0.70317 to 0.70138, saving model to weights-improvement-best-model.hdf5
Epoch 60/500
 - 3s - loss: 0.6953 - acc: 0.7710 - val_loss: 0.6997 - val_acc: 0.7684

Epoch 00060: val_loss improved from 0.70138 to 0.69966, saving model to weights-improvement-best-model.hdf5
Epoch 61/500
 - 3s - loss: 0.6936 - acc: 0.7703 - val_loss: 0.7254 - val_acc: 0.7497

Epoch 00061: val_loss did not improve from 0.69966
Epoch 62/500
 - 3s - loss: 0.7219 - acc: 0.7545 - val_loss: 0.7068 - val_acc: 0.7646

Epoch 00062: val_loss did not improve from 0.69966
Epoch 63/500
 - 3s - loss: 0.6949 - acc: 0.7694 - val_loss: 0.6936 - val_acc: 0.7716

Epoch 00063: val_loss improved from 0.69966 to 0.69362, saving model to weights-improvement-best-model.hdf5
Epoch 64/500
 - 3s - loss: 0.6897 - acc: 0.7712 - val_loss: 0.7014 - val_acc: 0.7650

Epoch 00064: val_loss did not improve from 0.69362
Epoch 65/500
 - 3s - loss: 0.7130 - acc: 0.7574 - val_loss: 0.6948 - val_acc: 0.7692

Epoch 00065: val_loss did not improve from 0.69362
Epoch 66/500
 - 3s - loss: 0.6884 - acc: 0.7716 - val_loss: 0.6958 - val_acc: 0.7652

Epoch 00066: val_loss did not improve from 0.69362
Epoch 67/500
 - 3s - loss: 0.6937 - acc: 0.7693 - val_loss: 0.7088 - val_acc: 0.7604

Epoch 00067: val_loss did not improve from 0.69362
Epoch 68/500
 - 3s - loss: 0.7003 - acc: 0.7623 - val_loss: 0.6924 - val_acc: 0.7652

Epoch 00068: val_loss improved from 0.69362 to 0.69243, saving model to weights-improvement-best-model.hdf5
Epoch 69/500
 - 3s - loss: 0.6903 - acc: 0.7702 - val_loss: 0.6992 - val_acc: 0.7652

Epoch 00069: val_loss did not improve from 0.69243
Epoch 70/500
 - 3s - loss: 0.6929 - acc: 0.7680 - val_loss: 0.6939 - val_acc: 0.7714

Epoch 00070: val_loss did not improve from 0.69243
Epoch 71/500
 - 3s - loss: 0.6879 - acc: 0.7725 - val_loss: 0.7020 - val_acc: 0.7657

Epoch 00071: val_loss did not improve from 0.69243
Epoch 72/500
 - 3s - loss: 0.6853 - acc: 0.7742 - val_loss: 0.7177 - val_acc: 0.7623

Epoch 00072: val_loss did not improve from 0.69243
Epoch 73/500
 - 3s - loss: 0.7134 - acc: 0.7582 - val_loss: 0.6913 - val_acc: 0.7685

Epoch 00073: val_loss improved from 0.69243 to 0.69134, saving model to weights-improvement-best-model.hdf5
Epoch 74/500
 - 3s - loss: 0.6820 - acc: 0.7763 - val_loss: 0.6881 - val_acc: 0.7704

Epoch 00074: val_loss improved from 0.69134 to 0.68813, saving model to weights-improvement-best-model.hdf5
Epoch 75/500
 - 3s - loss: 0.6792 - acc: 0.7772 - val_loss: 0.6908 - val_acc: 0.7720

Epoch 00075: val_loss did not improve from 0.68813
Epoch 76/500
 - 3s - loss: 0.6810 - acc: 0.7783 - val_loss: 0.7487 - val_acc: 0.7525

Epoch 00076: val_loss did not improve from 0.68813
Epoch 77/500
 - 3s - loss: 0.7158 - acc: 0.7579 - val_loss: 0.6872 - val_acc: 0.7716

Epoch 00077: val_loss improved from 0.68813 to 0.68725, saving model to weights-improvement-best-model.hdf5
Epoch 78/500
 - 3s - loss: 0.6772 - acc: 0.7780 - val_loss: 0.6892 - val_acc: 0.7705

Epoch 00078: val_loss did not improve from 0.68725
Epoch 79/500
 - 3s - loss: 0.6857 - acc: 0.7737 - val_loss: 0.7127 - val_acc: 0.7586

Epoch 00079: val_loss did not improve from 0.68725
Epoch 80/500
 - 3s - loss: 0.6923 - acc: 0.7701 - val_loss: 0.6839 - val_acc: 0.7699

Epoch 00080: val_loss improved from 0.68725 to 0.68388, saving model to weights-improvement-best-model.hdf5
Epoch 81/500
 - 3s - loss: 0.6785 - acc: 0.7771 - val_loss: 0.6947 - val_acc: 0.7650

Epoch 00081: val_loss did not improve from 0.68388
Epoch 82/500
 - 3s - loss: 0.6827 - acc: 0.7761 - val_loss: 0.7019 - val_acc: 0.7670

Epoch 00082: val_loss did not improve from 0.68388
Epoch 83/500
 - 3s - loss: 0.6888 - acc: 0.7734 - val_loss: 0.7127 - val_acc: 0.7509

Epoch 00083: val_loss did not improve from 0.68388
Epoch 84/500
 - 3s - loss: 0.6878 - acc: 0.7714 - val_loss: 0.6932 - val_acc: 0.7743

Epoch 00084: val_loss did not improve from 0.68388
Epoch 85/500
 - 3s - loss: 0.6845 - acc: 0.7749 - val_loss: 0.6936 - val_acc: 0.7704

Epoch 00085: val_loss did not improve from 0.68388
Epoch 86/500
 - 3s - loss: 0.6791 - acc: 0.7781 - val_loss: 0.6901 - val_acc: 0.7724

Epoch 00086: val_loss did not improve from 0.68388
Epoch 87/500
 - 3s - loss: 0.6756 - acc: 0.7814 - val_loss: 0.6958 - val_acc: 0.7677

Epoch 00087: val_loss did not improve from 0.68388
Epoch 88/500
 - 3s - loss: 0.6811 - acc: 0.7771 - val_loss: 0.7309 - val_acc: 0.7433

Epoch 00088: val_loss did not improve from 0.68388
Epoch 89/500
 - 3s - loss: 0.6937 - acc: 0.7696 - val_loss: 0.6822 - val_acc: 0.7772

Epoch 00089: val_loss improved from 0.68388 to 0.68222, saving model to weights-improvement-best-model.hdf5
Epoch 90/500
 - 3s - loss: 0.6707 - acc: 0.7828 - val_loss: 0.7141 - val_acc: 0.7549

Epoch 00090: val_loss did not improve from 0.68222
Epoch 91/500
 - 3s - loss: 0.6863 - acc: 0.7747 - val_loss: 0.7110 - val_acc: 0.7629

Epoch 00091: val_loss did not improve from 0.68222
Epoch 92/500
 - 3s - loss: 0.6825 - acc: 0.7767 - val_loss: 0.6983 - val_acc: 0.7611

Epoch 00092: val_loss did not improve from 0.68222
Epoch 93/500
 - 3s - loss: 0.6767 - acc: 0.7769 - val_loss: 0.6908 - val_acc: 0.7718

Epoch 00093: val_loss did not improve from 0.68222
Epoch 94/500
 - 3s - loss: 0.6771 - acc: 0.7793 - val_loss: 0.7015 - val_acc: 0.7626

Epoch 00094: val_loss did not improve from 0.68222
Epoch 95/500
 - 3s - loss: 0.6831 - acc: 0.7748 - val_loss: 0.7190 - val_acc: 0.7599

Epoch 00095: val_loss did not improve from 0.68222
Epoch 96/500
 - 3s - loss: 0.6805 - acc: 0.7784 - val_loss: 0.6826 - val_acc: 0.7768

Epoch 00096: val_loss did not improve from 0.68222
Epoch 97/500
 - 3s - loss: 0.6650 - acc: 0.7862 - val_loss: 0.6810 - val_acc: 0.7749

Epoch 00097: val_loss improved from 0.68222 to 0.68103, saving model to weights-improvement-best-model.hdf5
Epoch 98/500
 - 3s - loss: 0.6780 - acc: 0.7790 - val_loss: 0.6922 - val_acc: 0.7720

Epoch 00098: val_loss did not improve from 0.68103
Epoch 99/500
 - 3s - loss: 0.6731 - acc: 0.7817 - val_loss: 0.6856 - val_acc: 0.7739

Epoch 00099: val_loss did not improve from 0.68103
Epoch 100/500
 - 3s - loss: 0.6656 - acc: 0.7860 - val_loss: 0.6900 - val_acc: 0.7704

Epoch 00100: val_loss did not improve from 0.68103
Epoch 101/500
 - 3s - loss: 0.6793 - acc: 0.7772 - val_loss: 0.7150 - val_acc: 0.7583

Epoch 00101: val_loss did not improve from 0.68103
Epoch 102/500
 - 3s - loss: 0.6792 - acc: 0.7776 - val_loss: 0.6802 - val_acc: 0.7798

Epoch 00102: val_loss improved from 0.68103 to 0.68016, saving model to weights-improvement-best-model.hdf5
Epoch 103/500
 - 3s - loss: 0.6629 - acc: 0.7861 - val_loss: 0.6899 - val_acc: 0.7721

Epoch 00103: val_loss did not improve from 0.68016
Epoch 104/500
 - 3s - loss: 0.6773 - acc: 0.7778 - val_loss: 0.6950 - val_acc: 0.7672

Epoch 00104: val_loss did not improve from 0.68016
Epoch 105/500
 - 3s - loss: 0.6660 - acc: 0.7839 - val_loss: 0.6814 - val_acc: 0.7789

Epoch 00105: val_loss did not improve from 0.68016
Epoch 106/500
 - 3s - loss: 0.6630 - acc: 0.7865 - val_loss: 0.6790 - val_acc: 0.7784

Epoch 00106: val_loss improved from 0.68016 to 0.67900, saving model to weights-improvement-best-model.hdf5
Epoch 107/500
 - 3s - loss: 0.6786 - acc: 0.7768 - val_loss: 0.7279 - val_acc: 0.7592

Epoch 00107: val_loss did not improve from 0.67900
Epoch 108/500
 - 3s - loss: 0.6789 - acc: 0.7795 - val_loss: 0.6912 - val_acc: 0.7709

Epoch 00108: val_loss did not improve from 0.67900
Epoch 109/500
 - 3s - loss: 0.6684 - acc: 0.7835 - val_loss: 0.6802 - val_acc: 0.7786

Epoch 00109: val_loss did not improve from 0.67900
Epoch 110/500
 - 3s - loss: 0.6699 - acc: 0.7825 - val_loss: 0.7052 - val_acc: 0.7574

Epoch 00110: val_loss did not improve from 0.67900
Epoch 111/500
 - 3s - loss: 0.6739 - acc: 0.7802 - val_loss: 0.6870 - val_acc: 0.7722

Epoch 00111: val_loss did not improve from 0.67900
Epoch 112/500
 - 3s - loss: 0.6636 - acc: 0.7839 - val_loss: 0.7160 - val_acc: 0.7544

Epoch 00112: val_loss did not improve from 0.67900
Epoch 113/500
 - 3s - loss: 0.6774 - acc: 0.7785 - val_loss: 0.6849 - val_acc: 0.7762

Epoch 00113: val_loss did not improve from 0.67900
Epoch 114/500
 - 3s - loss: 0.6630 - acc: 0.7874 - val_loss: 0.6919 - val_acc: 0.7709

Epoch 00114: val_loss did not improve from 0.67900
Epoch 115/500
 - 3s - loss: 0.6835 - acc: 0.7730 - val_loss: 0.6951 - val_acc: 0.7668

Epoch 00115: val_loss did not improve from 0.67900
Epoch 116/500
 - 3s - loss: 0.6653 - acc: 0.7856 - val_loss: 0.7036 - val_acc: 0.7705

Epoch 00116: val_loss did not improve from 0.67900
Epoch 117/500
 - 3s - loss: 0.6746 - acc: 0.7817 - val_loss: 0.6810 - val_acc: 0.7736

Epoch 00117: val_loss did not improve from 0.67900
Epoch 118/500
 - 3s - loss: 0.6599 - acc: 0.7891 - val_loss: 0.6785 - val_acc: 0.7760

Epoch 00118: val_loss improved from 0.67900 to 0.67849, saving model to weights-improvement-best-model.hdf5
Epoch 119/500
 - 3s - loss: 0.6555 - acc: 0.7893 - val_loss: 0.6850 - val_acc: 0.7742

Epoch 00119: val_loss did not improve from 0.67849
Epoch 120/500
 - 3s - loss: 0.6691 - acc: 0.7813 - val_loss: 0.7019 - val_acc: 0.7657

Epoch 00120: val_loss did not improve from 0.67849
Epoch 121/500
 - 3s - loss: 0.6687 - acc: 0.7835 - val_loss: 0.6833 - val_acc: 0.7761

Epoch 00121: val_loss did not improve from 0.67849
Epoch 122/500
 - 3s - loss: 0.6616 - acc: 0.7843 - val_loss: 0.6879 - val_acc: 0.7702

Epoch 00122: val_loss did not improve from 0.67849
Epoch 123/500
 - 3s - loss: 0.6608 - acc: 0.7856 - val_loss: 0.6904 - val_acc: 0.7755

Epoch 00123: val_loss did not improve from 0.67849
Epoch 124/500
 - 3s - loss: 0.6629 - acc: 0.7842 - val_loss: 0.6777 - val_acc: 0.7764

Epoch 00124: val_loss improved from 0.67849 to 0.67774, saving model to weights-improvement-best-model.hdf5
Epoch 125/500
 - 3s - loss: 0.6525 - acc: 0.7931 - val_loss: 0.6864 - val_acc: 0.7721

Epoch 00125: val_loss did not improve from 0.67774
Epoch 126/500
 - 3s - loss: 0.6651 - acc: 0.7860 - val_loss: 0.7114 - val_acc: 0.7530

Epoch 00126: val_loss did not improve from 0.67774
Epoch 127/500
 - 3s - loss: 0.6716 - acc: 0.7809 - val_loss: 0.6854 - val_acc: 0.7754

Epoch 00127: val_loss did not improve from 0.67774
Epoch 128/500
 - 3s - loss: 0.6568 - acc: 0.7911 - val_loss: 0.6940 - val_acc: 0.7716

Epoch 00128: val_loss did not improve from 0.67774
Epoch 129/500
 - 3s - loss: 0.6747 - acc: 0.7783 - val_loss: 0.7097 - val_acc: 0.7554

Epoch 00129: val_loss did not improve from 0.67774
Epoch 130/500
 - 3s - loss: 0.6652 - acc: 0.7840 - val_loss: 0.6791 - val_acc: 0.7755

Epoch 00130: val_loss did not improve from 0.67774
Epoch 131/500
 - 3s - loss: 0.6514 - acc: 0.7933 - val_loss: 0.6773 - val_acc: 0.7744

Epoch 00131: val_loss improved from 0.67774 to 0.67731, saving model to weights-improvement-best-model.hdf5
Epoch 132/500
 - 3s - loss: 0.6538 - acc: 0.7920 - val_loss: 0.7346 - val_acc: 0.7592

Epoch 00132: val_loss did not improve from 0.67731
Epoch 133/500
 - 3s - loss: 0.6856 - acc: 0.7750 - val_loss: 0.6769 - val_acc: 0.7790

Epoch 00133: val_loss improved from 0.67731 to 0.67688, saving model to weights-improvement-best-model.hdf5
Epoch 134/500
 - 3s - loss: 0.6515 - acc: 0.7967 - val_loss: 0.6827 - val_acc: 0.7721

Epoch 00134: val_loss did not improve from 0.67688
Epoch 135/500
 - 3s - loss: 0.6523 - acc: 0.7919 - val_loss: 0.6906 - val_acc: 0.7775

Epoch 00135: val_loss did not improve from 0.67688
Epoch 136/500
 - 3s - loss: 0.6708 - acc: 0.7819 - val_loss: 0.6772 - val_acc: 0.7829

Epoch 00136: val_loss did not improve from 0.67688
Epoch 137/500
 - 3s - loss: 0.6500 - acc: 0.7942 - val_loss: 0.6771 - val_acc: 0.7801

Epoch 00137: val_loss did not improve from 0.67688
Epoch 138/500
 - 3s - loss: 0.6649 - acc: 0.7851 - val_loss: 0.7093 - val_acc: 0.7650

Epoch 00138: val_loss did not improve from 0.67688
Epoch 139/500
 - 3s - loss: 0.6624 - acc: 0.7871 - val_loss: 0.7079 - val_acc: 0.7557

Epoch 00139: val_loss did not improve from 0.67688
Epoch 140/500
 - 3s - loss: 0.6692 - acc: 0.7821 - val_loss: 0.6798 - val_acc: 0.7812

Epoch 00140: val_loss did not improve from 0.67688
Epoch 141/500
 - 3s - loss: 0.6498 - acc: 0.7931 - val_loss: 0.6884 - val_acc: 0.7658

Epoch 00141: val_loss did not improve from 0.67688
Epoch 142/500
 - 3s - loss: 0.6656 - acc: 0.7857 - val_loss: 0.6868 - val_acc: 0.7759

Epoch 00142: val_loss did not improve from 0.67688
Epoch 143/500
 - 3s - loss: 0.6505 - acc: 0.7945 - val_loss: 0.6839 - val_acc: 0.7757

Epoch 00143: val_loss did not improve from 0.67688
Epoch 144/500
 - 3s - loss: 0.6675 - acc: 0.7836 - val_loss: 0.7076 - val_acc: 0.7573

Epoch 00144: val_loss did not improve from 0.67688
Epoch 145/500
 - 3s - loss: 0.6593 - acc: 0.7897 - val_loss: 0.6774 - val_acc: 0.7789

Epoch 00145: val_loss did not improve from 0.67688
Epoch 146/500
 - 3s - loss: 0.6468 - acc: 0.7950 - val_loss: 0.6780 - val_acc: 0.7791

Epoch 00146: val_loss did not improve from 0.67688
Epoch 147/500
 - 3s - loss: 0.6634 - acc: 0.7837 - val_loss: 0.6771 - val_acc: 0.7813

Epoch 00147: val_loss did not improve from 0.67688
Epoch 148/500
 - 3s - loss: 0.6493 - acc: 0.7943 - val_loss: 0.6780 - val_acc: 0.7741

Epoch 00148: val_loss did not improve from 0.67688
Epoch 149/500
 - 3s - loss: 0.6494 - acc: 0.7937 - val_loss: 0.7406 - val_acc: 0.7612

Epoch 00149: val_loss did not improve from 0.67688
Epoch 150/500
 - 3s - loss: 0.6723 - acc: 0.7845 - val_loss: 0.7060 - val_acc: 0.7583

Epoch 00150: val_loss did not improve from 0.67688
Epoch 151/500
 - 3s - loss: 0.6636 - acc: 0.7864 - val_loss: 0.6923 - val_acc: 0.7734

Epoch 00151: val_loss did not improve from 0.67688
Epoch 152/500
 - 3s - loss: 0.6513 - acc: 0.7932 - val_loss: 0.6851 - val_acc: 0.7780

Epoch 00152: val_loss did not improve from 0.67688
Epoch 153/500
 - 3s - loss: 0.6462 - acc: 0.7978 - val_loss: 0.7064 - val_acc: 0.7710

Epoch 00153: val_loss did not improve from 0.67688
Epoch 154/500
 - 3s - loss: 0.6728 - acc: 0.7813 - val_loss: 0.6810 - val_acc: 0.7738

Epoch 00154: val_loss did not improve from 0.67688
Epoch 155/500
 - 3s - loss: 0.6481 - acc: 0.7971 - val_loss: 0.6882 - val_acc: 0.7763

Epoch 00155: val_loss did not improve from 0.67688
Epoch 156/500
 - 3s - loss: 0.6634 - acc: 0.7867 - val_loss: 0.6849 - val_acc: 0.7791

Epoch 00156: val_loss did not improve from 0.67688
Epoch 157/500
 - 3s - loss: 0.6493 - acc: 0.7973 - val_loss: 0.6844 - val_acc: 0.7731

Epoch 00157: val_loss did not improve from 0.67688
Epoch 158/500
 - 3s - loss: 0.6522 - acc: 0.7899 - val_loss: 0.7029 - val_acc: 0.7607

Epoch 00158: val_loss did not improve from 0.67688
Epoch 159/500
 - 3s - loss: 0.6599 - acc: 0.7882 - val_loss: 0.6785 - val_acc: 0.7790

Epoch 00159: val_loss did not improve from 0.67688
Epoch 160/500
 - 3s - loss: 0.6462 - acc: 0.7977 - val_loss: 0.6825 - val_acc: 0.7723

Epoch 00160: val_loss did not improve from 0.67688
Epoch 161/500
 - 3s - loss: 0.6649 - acc: 0.7843 - val_loss: 0.6854 - val_acc: 0.7728

Epoch 00161: val_loss did not improve from 0.67688
Epoch 162/500
 - 3s - loss: 0.6450 - acc: 0.7991 - val_loss: 0.6782 - val_acc: 0.7799

Epoch 00162: val_loss did not improve from 0.67688
Epoch 163/500
 - 3s - loss: 0.6427 - acc: 0.7987 - val_loss: 0.6958 - val_acc: 0.7723

Epoch 00163: val_loss did not improve from 0.67688
Epoch 164/500
 - 3s - loss: 0.6795 - acc: 0.7772 - val_loss: 0.6712 - val_acc: 0.7795

Epoch 00164: val_loss improved from 0.67688 to 0.67119, saving model to weights-improvement-best-model.hdf5
Epoch 165/500
 - 3s - loss: 0.6432 - acc: 0.7982 - val_loss: 0.6875 - val_acc: 0.7766

Epoch 00165: val_loss did not improve from 0.67119
Epoch 166/500
 - 3s - loss: 0.6545 - acc: 0.7907 - val_loss: 0.6796 - val_acc: 0.7761

Epoch 00166: val_loss did not improve from 0.67119
Epoch 167/500
 - 3s - loss: 0.6486 - acc: 0.7933 - val_loss: 0.6914 - val_acc: 0.7774

Epoch 00167: val_loss did not improve from 0.67119
Epoch 168/500
 - 3s - loss: 0.6538 - acc: 0.7910 - val_loss: 0.6865 - val_acc: 0.7775

Epoch 00168: val_loss did not improve from 0.67119
Epoch 169/500
 - 3s - loss: 0.6519 - acc: 0.7932 - val_loss: 0.6824 - val_acc: 0.7784

Epoch 00169: val_loss did not improve from 0.67119
Epoch 170/500
 - 3s - loss: 0.6493 - acc: 0.7928 - val_loss: 0.6925 - val_acc: 0.7634

Epoch 00170: val_loss did not improve from 0.67119
Epoch 171/500
 - 3s - loss: 0.6542 - acc: 0.7902 - val_loss: 0.7087 - val_acc: 0.7704

Epoch 00171: val_loss did not improve from 0.67119
Epoch 172/500
 - 3s - loss: 0.6526 - acc: 0.7928 - val_loss: 0.7007 - val_acc: 0.7625

Epoch 00172: val_loss did not improve from 0.67119
Epoch 173/500
 - 3s - loss: 0.6572 - acc: 0.7876 - val_loss: 0.6781 - val_acc: 0.7817

Epoch 00173: val_loss did not improve from 0.67119
Epoch 174/500
 - 3s - loss: 0.6379 - acc: 0.8015 - val_loss: 0.6865 - val_acc: 0.7734

Epoch 00174: val_loss did not improve from 0.67119
Epoch 175/500
 - 3s - loss: 0.6559 - acc: 0.7883 - val_loss: 0.6737 - val_acc: 0.7817

Epoch 00175: val_loss did not improve from 0.67119
Epoch 176/500
 - 3s - loss: 0.6401 - acc: 0.7994 - val_loss: 0.6948 - val_acc: 0.7723

Epoch 00176: val_loss did not improve from 0.67119
Epoch 177/500
 - 3s - loss: 0.6610 - acc: 0.7871 - val_loss: 0.6923 - val_acc: 0.7702

Epoch 00177: val_loss did not improve from 0.67119
Epoch 178/500
 - 3s - loss: 0.6484 - acc: 0.7958 - val_loss: 0.6840 - val_acc: 0.7750

Epoch 00178: val_loss did not improve from 0.67119
Epoch 179/500
 - 3s - loss: 0.6412 - acc: 0.7987 - val_loss: 0.6823 - val_acc: 0.7803

Epoch 00179: val_loss did not improve from 0.67119
Epoch 180/500
 - 3s - loss: 0.6623 - acc: 0.7862 - val_loss: 0.6829 - val_acc: 0.7787

Epoch 00180: val_loss did not improve from 0.67119
Epoch 181/500
 - 3s - loss: 0.6425 - acc: 0.7966 - val_loss: 0.6835 - val_acc: 0.7815

Epoch 00181: val_loss did not improve from 0.67119
Epoch 182/500
 - 3s - loss: 0.6441 - acc: 0.7950 - val_loss: 0.6781 - val_acc: 0.7791

Epoch 00182: val_loss did not improve from 0.67119
Epoch 183/500
 - 3s - loss: 0.6432 - acc: 0.7968 - val_loss: 0.6849 - val_acc: 0.7743

Epoch 00183: val_loss did not improve from 0.67119
Epoch 184/500
 - 3s - loss: 0.6473 - acc: 0.7957 - val_loss: 0.7240 - val_acc: 0.7671

Epoch 00184: val_loss did not improve from 0.67119
Epoch 185/500
 - 3s - loss: 0.6598 - acc: 0.7882 - val_loss: 0.6770 - val_acc: 0.7766

Epoch 00185: val_loss did not improve from 0.67119
Epoch 186/500
 - 3s - loss: 0.6465 - acc: 0.7959 - val_loss: 0.6883 - val_acc: 0.7743

Epoch 00186: val_loss did not improve from 0.67119
Epoch 187/500
 - 3s - loss: 0.6418 - acc: 0.7977 - val_loss: 0.6901 - val_acc: 0.7703

Epoch 00187: val_loss did not improve from 0.67119
Epoch 188/500
 - 3s - loss: 0.6435 - acc: 0.7954 - val_loss: 0.6928 - val_acc: 0.7748

Epoch 00188: val_loss did not improve from 0.67119
Epoch 189/500
 - 3s - loss: 0.6528 - acc: 0.7930 - val_loss: 0.6787 - val_acc: 0.7758

Epoch 00189: val_loss did not improve from 0.67119
Epoch 190/500
 - 3s - loss: 0.6374 - acc: 0.8012 - val_loss: 0.6826 - val_acc: 0.7786

Epoch 00190: val_loss did not improve from 0.67119
Epoch 191/500
 - 3s - loss: 0.6359 - acc: 0.8029 - val_loss: 0.6848 - val_acc: 0.7824

Epoch 00191: val_loss did not improve from 0.67119
Epoch 192/500
 - 3s - loss: 0.6391 - acc: 0.8022 - val_loss: 0.6773 - val_acc: 0.7780

Epoch 00192: val_loss did not improve from 0.67119
Epoch 193/500
 - 3s - loss: 0.6759 - acc: 0.7783 - val_loss: 0.6853 - val_acc: 0.7742

Epoch 00193: val_loss did not improve from 0.67119
Epoch 194/500
 - 3s - loss: 0.6390 - acc: 0.7996 - val_loss: 0.6795 - val_acc: 0.7770

Epoch 00194: val_loss did not improve from 0.67119
Epoch 195/500
 - 3s - loss: 0.6400 - acc: 0.7979 - val_loss: 0.7035 - val_acc: 0.7732

Epoch 00195: val_loss did not improve from 0.67119
Epoch 196/500
 - 3s - loss: 0.6561 - acc: 0.7898 - val_loss: 0.6977 - val_acc: 0.7666

Epoch 00196: val_loss did not improve from 0.67119
Epoch 197/500
 - 3s - loss: 0.6433 - acc: 0.7958 - val_loss: 0.6798 - val_acc: 0.7774

Epoch 00197: val_loss did not improve from 0.67119
Epoch 198/500
 - 3s - loss: 0.6379 - acc: 0.8005 - val_loss: 0.6847 - val_acc: 0.7749

Epoch 00198: val_loss did not improve from 0.67119
Epoch 199/500
 - 3s - loss: 0.6481 - acc: 0.7922 - val_loss: 0.6840 - val_acc: 0.7742

Epoch 00199: val_loss did not improve from 0.67119
Epoch 200/500
 - 3s - loss: 0.6344 - acc: 0.8012 - val_loss: 0.6901 - val_acc: 0.7754

Epoch 00200: val_loss did not improve from 0.67119
Epoch 201/500
 - 3s - loss: 0.6584 - acc: 0.7887 - val_loss: 0.6753 - val_acc: 0.7830

Epoch 00201: val_loss did not improve from 0.67119
Epoch 202/500
 - 3s - loss: 0.6371 - acc: 0.8003 - val_loss: 0.6886 - val_acc: 0.7780

Epoch 00202: val_loss did not improve from 0.67119
Epoch 203/500
 - 3s - loss: 0.6528 - acc: 0.7920 - val_loss: 0.6736 - val_acc: 0.7846

Epoch 00203: val_loss did not improve from 0.67119
Epoch 204/500
 - 3s - loss: 0.6341 - acc: 0.8045 - val_loss: 0.6985 - val_acc: 0.7691

Epoch 00204: val_loss did not improve from 0.67119
Epoch 205/500
 - 3s - loss: 0.6527 - acc: 0.7925 - val_loss: 0.6900 - val_acc: 0.7646

Epoch 00205: val_loss did not improve from 0.67119
Epoch 206/500
 - 3s - loss: 0.6498 - acc: 0.7896 - val_loss: 0.6947 - val_acc: 0.7769

Epoch 00206: val_loss did not improve from 0.67119
Epoch 207/500
 - 3s - loss: 0.6412 - acc: 0.7997 - val_loss: 0.6727 - val_acc: 0.7785

Epoch 00207: val_loss did not improve from 0.67119
Epoch 208/500
 - 3s - loss: 0.6320 - acc: 0.8061 - val_loss: 0.6858 - val_acc: 0.7755

Epoch 00208: val_loss did not improve from 0.67119
Epoch 209/500
 - 3s - loss: 0.6339 - acc: 0.8027 - val_loss: 0.7185 - val_acc: 0.7692

Epoch 00209: val_loss did not improve from 0.67119
Epoch 210/500
 - 3s - loss: 0.6715 - acc: 0.7808 - val_loss: 0.6726 - val_acc: 0.7825

Epoch 00210: val_loss did not improve from 0.67119
Epoch 211/500
 - 3s - loss: 0.6307 - acc: 0.8045 - val_loss: 0.6795 - val_acc: 0.7839

Epoch 00211: val_loss did not improve from 0.67119
Epoch 212/500
 - 3s - loss: 0.6393 - acc: 0.7986 - val_loss: 0.6887 - val_acc: 0.7732

Epoch 00212: val_loss did not improve from 0.67119
Epoch 213/500
 - 3s - loss: 0.6496 - acc: 0.7947 - val_loss: 0.6737 - val_acc: 0.7828

Epoch 00213: val_loss did not improve from 0.67119
Epoch 214/500
 - 3s - loss: 0.6324 - acc: 0.8044 - val_loss: 0.6696 - val_acc: 0.7800

Epoch 00214: val_loss improved from 0.67119 to 0.66956, saving model to weights-improvement-best-model.hdf5
Epoch 215/500
 - 3s - loss: 0.6309 - acc: 0.8052 - val_loss: 0.6847 - val_acc: 0.7713

Epoch 00215: val_loss did not improve from 0.66956
Epoch 216/500
 - 3s - loss: 0.6629 - acc: 0.7870 - val_loss: 0.7237 - val_acc: 0.7683

Epoch 00216: val_loss did not improve from 0.66956
Epoch 217/500
 - 3s - loss: 0.6469 - acc: 0.7980 - val_loss: 0.6757 - val_acc: 0.7812

Epoch 00217: val_loss did not improve from 0.66956
Epoch 218/500
 - 3s - loss: 0.6331 - acc: 0.8029 - val_loss: 0.6745 - val_acc: 0.7773

Epoch 00218: val_loss did not improve from 0.66956
Epoch 219/500
 - 3s - loss: 0.6398 - acc: 0.7994 - val_loss: 0.6802 - val_acc: 0.7830

Epoch 00219: val_loss did not improve from 0.66956
Epoch 220/500
 - 3s - loss: 0.6373 - acc: 0.8005 - val_loss: 0.7087 - val_acc: 0.7517

Epoch 00220: val_loss did not improve from 0.66956
Epoch 221/500
 - 3s - loss: 0.6527 - acc: 0.7910 - val_loss: 0.6850 - val_acc: 0.7782

Epoch 00221: val_loss did not improve from 0.66956
Epoch 222/500
 - 3s - loss: 0.6330 - acc: 0.8040 - val_loss: 0.6838 - val_acc: 0.7778

Epoch 00222: val_loss did not improve from 0.66956
Epoch 223/500
 - 3s - loss: 0.6431 - acc: 0.7980 - val_loss: 0.6815 - val_acc: 0.7749

Epoch 00223: val_loss did not improve from 0.66956
Epoch 224/500
 - 3s - loss: 0.6356 - acc: 0.8027 - val_loss: 0.6942 - val_acc: 0.7746

Epoch 00224: val_loss did not improve from 0.66956
Epoch 225/500
 - 3s - loss: 0.6486 - acc: 0.7935 - val_loss: 0.6912 - val_acc: 0.7612

Epoch 00225: val_loss did not improve from 0.66956
Epoch 226/500
 - 3s - loss: 0.6381 - acc: 0.7990 - val_loss: 0.6918 - val_acc: 0.7740

Epoch 00226: val_loss did not improve from 0.66956
Epoch 227/500
 - 3s - loss: 0.6380 - acc: 0.8000 - val_loss: 0.6832 - val_acc: 0.7763

Epoch 00227: val_loss did not improve from 0.66956
Epoch 228/500
 - 3s - loss: 0.6398 - acc: 0.7980 - val_loss: 0.6830 - val_acc: 0.7787

Epoch 00228: val_loss did not improve from 0.66956
Epoch 229/500
 - 3s - loss: 0.6371 - acc: 0.7985 - val_loss: 0.6814 - val_acc: 0.7706

Epoch 00229: val_loss did not improve from 0.66956
Epoch 230/500
 - 4s - loss: 0.6426 - acc: 0.7985 - val_loss: 0.6818 - val_acc: 0.7795

Epoch 00230: val_loss did not improve from 0.66956
Epoch 231/500
 - 3s - loss: 0.6313 - acc: 0.8060 - val_loss: 0.6734 - val_acc: 0.7811

Epoch 00231: val_loss did not improve from 0.66956
Epoch 232/500
 - 3s - loss: 0.6471 - acc: 0.7957 - val_loss: 0.6893 - val_acc: 0.7703

Epoch 00232: val_loss did not improve from 0.66956
Epoch 233/500
 - 3s - loss: 0.6312 - acc: 0.8064 - val_loss: 0.6817 - val_acc: 0.7825

Epoch 00233: val_loss did not improve from 0.66956
Epoch 234/500
 - 3s - loss: 0.6348 - acc: 0.8044 - val_loss: 0.6956 - val_acc: 0.7667

Epoch 00234: val_loss did not improve from 0.66956
Epoch 235/500
 - 3s - loss: 0.6473 - acc: 0.7962 - val_loss: 0.6958 - val_acc: 0.7786

Epoch 00235: val_loss did not improve from 0.66956
Epoch 236/500
 - 3s - loss: 0.6464 - acc: 0.7957 - val_loss: 0.6835 - val_acc: 0.7741

Epoch 00236: val_loss did not improve from 0.66956
Epoch 237/500
 - 3s - loss: 0.6328 - acc: 0.8031 - val_loss: 0.6922 - val_acc: 0.7757

Epoch 00237: val_loss did not improve from 0.66956
Epoch 238/500
 - 3s - loss: 0.6342 - acc: 0.8057 - val_loss: 0.6855 - val_acc: 0.7733

Epoch 00238: val_loss did not improve from 0.66956
Epoch 239/500
 - 3s - loss: 0.6476 - acc: 0.7928 - val_loss: 0.6780 - val_acc: 0.7787

Epoch 00239: val_loss did not improve from 0.66956
Epoch 240/500
 - 3s - loss: 0.6270 - acc: 0.8062 - val_loss: 0.6869 - val_acc: 0.7771

Epoch 00240: val_loss did not improve from 0.66956
Epoch 241/500
 - 3s - loss: 0.6420 - acc: 0.7965 - val_loss: 0.7095 - val_acc: 0.7703

Epoch 00241: val_loss did not improve from 0.66956
Epoch 242/500
 - 3s - loss: 0.6385 - acc: 0.8021 - val_loss: 0.6807 - val_acc: 0.7795

Epoch 00242: val_loss did not improve from 0.66956
Epoch 243/500
 - 3s - loss: 0.6284 - acc: 0.8052 - val_loss: 0.6933 - val_acc: 0.7769

Epoch 00243: val_loss did not improve from 0.66956
Epoch 244/500
 - 3s - loss: 0.6435 - acc: 0.7970 - val_loss: 0.6946 - val_acc: 0.7629

Epoch 00244: val_loss did not improve from 0.66956
Epoch 245/500
 - 3s - loss: 0.6373 - acc: 0.8019 - val_loss: 0.6802 - val_acc: 0.7775

Epoch 00245: val_loss did not improve from 0.66956
Epoch 246/500
 - 3s - loss: 0.6367 - acc: 0.8014 - val_loss: 0.6936 - val_acc: 0.7672

Epoch 00246: val_loss did not improve from 0.66956
Epoch 247/500
 - 3s - loss: 0.6322 - acc: 0.8046 - val_loss: 0.6822 - val_acc: 0.7829

Epoch 00247: val_loss did not improve from 0.66956
Epoch 248/500
 - 3s - loss: 0.6353 - acc: 0.8016 - val_loss: 0.6898 - val_acc: 0.7714

Epoch 00248: val_loss did not improve from 0.66956
Epoch 249/500
 - 3s - loss: 0.6358 - acc: 0.8003 - val_loss: 0.6774 - val_acc: 0.7807

Epoch 00249: val_loss did not improve from 0.66956
Epoch 250/500
 - 3s - loss: 0.6274 - acc: 0.8043 - val_loss: 0.6746 - val_acc: 0.7827

Epoch 00250: val_loss did not improve from 0.66956
Epoch 251/500
 - 3s - loss: 0.6212 - acc: 0.8077 - val_loss: 0.6931 - val_acc: 0.7666

Epoch 00251: val_loss did not improve from 0.66956
Epoch 252/500
 - 3s - loss: 0.6608 - acc: 0.7881 - val_loss: 0.6938 - val_acc: 0.7782

Epoch 00252: val_loss did not improve from 0.66956
Epoch 253/500
 - 3s - loss: 0.6291 - acc: 0.8054 - val_loss: 0.6769 - val_acc: 0.7780

Epoch 00253: val_loss did not improve from 0.66956
Epoch 254/500
 - 3s - loss: 0.6247 - acc: 0.8090 - val_loss: 0.6919 - val_acc: 0.7751

Epoch 00254: val_loss did not improve from 0.66956
Epoch 255/500
 - 3s - loss: 0.6511 - acc: 0.7934 - val_loss: 0.6776 - val_acc: 0.7811

Epoch 00255: val_loss did not improve from 0.66956
Epoch 256/500
 - 3s - loss: 0.6258 - acc: 0.8091 - val_loss: 0.6738 - val_acc: 0.7856

Epoch 00256: val_loss did not improve from 0.66956
Epoch 257/500
 - 3s - loss: 0.6291 - acc: 0.8046 - val_loss: 0.6796 - val_acc: 0.7726

Epoch 00257: val_loss did not improve from 0.66956
Epoch 258/500
 - 3s - loss: 0.6341 - acc: 0.8031 - val_loss: 0.7047 - val_acc: 0.7720

Epoch 00258: val_loss did not improve from 0.66956
Epoch 259/500
 - 3s - loss: 0.6365 - acc: 0.7978 - val_loss: 0.6968 - val_acc: 0.7699

Epoch 00259: val_loss did not improve from 0.66956
Epoch 260/500
 - 3s - loss: 0.6332 - acc: 0.8014 - val_loss: 0.6914 - val_acc: 0.7809

Epoch 00260: val_loss did not improve from 0.66956
Epoch 261/500
 - 3s - loss: 0.6311 - acc: 0.8036 - val_loss: 0.6966 - val_acc: 0.7666

Epoch 00261: val_loss did not improve from 0.66956
Epoch 262/500
 - 3s - loss: 0.6433 - acc: 0.7961 - val_loss: 0.6844 - val_acc: 0.7772

Epoch 00262: val_loss did not improve from 0.66956
Epoch 263/500
 - 3s - loss: 0.6322 - acc: 0.8026 - val_loss: 0.6807 - val_acc: 0.7733

Epoch 00263: val_loss did not improve from 0.66956
Epoch 264/500
 - 3s - loss: 0.6253 - acc: 0.8061 - val_loss: 0.6793 - val_acc: 0.7819

Epoch 00264: val_loss did not improve from 0.66956
Epoch 265/500
 - 3s - loss: 0.6256 - acc: 0.8046 - val_loss: 0.6865 - val_acc: 0.7714

Epoch 00265: val_loss did not improve from 0.66956
Epoch 266/500
 - 3s - loss: 0.6446 - acc: 0.7948 - val_loss: 0.6989 - val_acc: 0.7806

Epoch 00266: val_loss did not improve from 0.66956
Epoch 267/500
 - 3s - loss: 0.6340 - acc: 0.8024 - val_loss: 0.7139 - val_acc: 0.7542

Epoch 00267: val_loss did not improve from 0.66956
Epoch 268/500
 - 3s - loss: 0.6397 - acc: 0.7963 - val_loss: 0.6818 - val_acc: 0.7818

Epoch 00268: val_loss did not improve from 0.66956
Epoch 269/500
 - 3s - loss: 0.6217 - acc: 0.8103 - val_loss: 0.6819 - val_acc: 0.7731

Epoch 00269: val_loss did not improve from 0.66956
Epoch 270/500
 - 3s - loss: 0.6358 - acc: 0.8017 - val_loss: 0.6972 - val_acc: 0.7760

Epoch 00270: val_loss did not improve from 0.66956
Epoch 271/500
 - 3s - loss: 0.6331 - acc: 0.8019 - val_loss: 0.6740 - val_acc: 0.7803

Epoch 00271: val_loss did not improve from 0.66956
Epoch 272/500
 - 3s - loss: 0.6245 - acc: 0.8063 - val_loss: 0.7027 - val_acc: 0.7748

Epoch 00272: val_loss did not improve from 0.66956
Epoch 273/500
 - 3s - loss: 0.6384 - acc: 0.7999 - val_loss: 0.7039 - val_acc: 0.7594

Epoch 00273: val_loss did not improve from 0.66956
Epoch 274/500
 - 3s - loss: 0.6358 - acc: 0.8019 - val_loss: 0.6735 - val_acc: 0.7827

Epoch 00274: val_loss did not improve from 0.66956
Epoch 275/500
 - 3s - loss: 0.6201 - acc: 0.8105 - val_loss: 0.6733 - val_acc: 0.7796

Epoch 00275: val_loss did not improve from 0.66956
Epoch 276/500
 - 3s - loss: 0.6191 - acc: 0.8096 - val_loss: 0.6817 - val_acc: 0.7748

Epoch 00276: val_loss did not improve from 0.66956
Epoch 277/500
 - 3s - loss: 0.6536 - acc: 0.7886 - val_loss: 0.6813 - val_acc: 0.7788

Epoch 00277: val_loss did not improve from 0.66956
Epoch 278/500
 - 3s - loss: 0.6207 - acc: 0.8099 - val_loss: 0.7231 - val_acc: 0.7496

Epoch 00278: val_loss did not improve from 0.66956
Epoch 279/500
 - 3s - loss: 0.6488 - acc: 0.7931 - val_loss: 0.6830 - val_acc: 0.7787

Epoch 00279: val_loss did not improve from 0.66956
Epoch 280/500
 - 3s - loss: 0.6200 - acc: 0.8104 - val_loss: 0.6757 - val_acc: 0.7800

Epoch 00280: val_loss did not improve from 0.66956
Epoch 281/500
 - 3s - loss: 0.6222 - acc: 0.8085 - val_loss: 0.6863 - val_acc: 0.7802

Epoch 00281: val_loss did not improve from 0.66956
Epoch 282/500
 - 3s - loss: 0.6317 - acc: 0.8035 - val_loss: 0.6997 - val_acc: 0.7647

Epoch 00282: val_loss did not improve from 0.66956
Epoch 283/500
 - 3s - loss: 0.6402 - acc: 0.7973 - val_loss: 0.6855 - val_acc: 0.7780

Epoch 00283: val_loss did not improve from 0.66956
Epoch 284/500
 - 3s - loss: 0.6289 - acc: 0.8035 - val_loss: 0.6759 - val_acc: 0.7794

Epoch 00284: val_loss did not improve from 0.66956
Epoch 285/500
 - 3s - loss: 0.6162 - acc: 0.8112 - val_loss: 0.6923 - val_acc: 0.7751

Epoch 00285: val_loss did not improve from 0.66956
Epoch 286/500
 - 3s - loss: 0.6372 - acc: 0.8019 - val_loss: 0.6984 - val_acc: 0.7736

Epoch 00286: val_loss did not improve from 0.66956
Epoch 287/500
 - 4s - loss: 0.6325 - acc: 0.8041 - val_loss: 0.6838 - val_acc: 0.7795

Epoch 00287: val_loss did not improve from 0.66956
Epoch 288/500
 - 3s - loss: 0.6240 - acc: 0.8057 - val_loss: 0.6725 - val_acc: 0.7821

Epoch 00288: val_loss did not improve from 0.66956
Epoch 289/500
 - 3s - loss: 0.6176 - acc: 0.8098 - val_loss: 0.6818 - val_acc: 0.7766

Epoch 00289: val_loss did not improve from 0.66956
Epoch 290/500
 - 3s - loss: 0.6371 - acc: 0.7991 - val_loss: 0.6899 - val_acc: 0.7812

Epoch 00290: val_loss did not improve from 0.66956
Epoch 291/500
 - 3s - loss: 0.6291 - acc: 0.8068 - val_loss: 0.6904 - val_acc: 0.7630

Epoch 00291: val_loss did not improve from 0.66956
Epoch 292/500
 - 3s - loss: 0.6370 - acc: 0.7988 - val_loss: 0.6798 - val_acc: 0.7796

Epoch 00292: val_loss did not improve from 0.66956
Epoch 293/500
 - 3s - loss: 0.6160 - acc: 0.8113 - val_loss: 0.6998 - val_acc: 0.7765

Epoch 00293: val_loss did not improve from 0.66956
Epoch 294/500
 - 3s - loss: 0.6298 - acc: 0.8026 - val_loss: 0.6956 - val_acc: 0.7763

Epoch 00294: val_loss did not improve from 0.66956
Epoch 295/500
 - 3s - loss: 0.6369 - acc: 0.7990 - val_loss: 0.6986 - val_acc: 0.7607

Epoch 00295: val_loss did not improve from 0.66956
Epoch 296/500
 - 3s - loss: 0.6262 - acc: 0.8040 - val_loss: 0.6705 - val_acc: 0.7830

Epoch 00296: val_loss did not improve from 0.66956
Epoch 297/500
 - 3s - loss: 0.6151 - acc: 0.8133 - val_loss: 0.6820 - val_acc: 0.7747

Epoch 00297: val_loss did not improve from 0.66956
Epoch 298/500
 - 3s - loss: 0.6324 - acc: 0.8023 - val_loss: 0.6956 - val_acc: 0.7622

Epoch 00298: val_loss did not improve from 0.66956
Epoch 299/500
 - 3s - loss: 0.6321 - acc: 0.8038 - val_loss: 0.6755 - val_acc: 0.7846

Epoch 00299: val_loss did not improve from 0.66956
Epoch 300/500
 - 3s - loss: 0.6250 - acc: 0.8062 - val_loss: 0.6771 - val_acc: 0.7779

Epoch 00300: val_loss did not improve from 0.66956
Epoch 301/500
 - 3s - loss: 0.6222 - acc: 0.8070 - val_loss: 0.6792 - val_acc: 0.7800

Epoch 00301: val_loss did not improve from 0.66956
Epoch 302/500
 - 3s - loss: 0.6200 - acc: 0.8070 - val_loss: 0.6897 - val_acc: 0.7723

Epoch 00302: val_loss did not improve from 0.66956
Epoch 303/500
 - 3s - loss: 0.6200 - acc: 0.8100 - val_loss: 0.6866 - val_acc: 0.7794

Epoch 00303: val_loss did not improve from 0.66956
Epoch 304/500
 - 3s - loss: 0.6413 - acc: 0.7956 - val_loss: 0.6727 - val_acc: 0.7816

Epoch 00304: val_loss did not improve from 0.66956
Epoch 305/500
 - 3s - loss: 0.6157 - acc: 0.8109 - val_loss: 0.6769 - val_acc: 0.7853

Epoch 00305: val_loss did not improve from 0.66956
Epoch 306/500
 - 3s - loss: 0.6276 - acc: 0.8034 - val_loss: 0.7038 - val_acc: 0.7697

Epoch 00306: val_loss did not improve from 0.66956
Epoch 307/500
 - 3s - loss: 0.6290 - acc: 0.8033 - val_loss: 0.6837 - val_acc: 0.7830

Epoch 00307: val_loss did not improve from 0.66956
Epoch 308/500
 - 3s - loss: 0.6161 - acc: 0.8107 - val_loss: 0.6838 - val_acc: 0.7782

Epoch 00308: val_loss did not improve from 0.66956
Epoch 309/500
 - 3s - loss: 0.6232 - acc: 0.8062 - val_loss: 0.6922 - val_acc: 0.7777

Epoch 00309: val_loss did not improve from 0.66956
Epoch 310/500
 - 3s - loss: 0.6287 - acc: 0.8044 - val_loss: 0.6848 - val_acc: 0.7769

Epoch 00310: val_loss did not improve from 0.66956
Epoch 311/500
 - 3s - loss: 0.6229 - acc: 0.8106 - val_loss: 0.6869 - val_acc: 0.7797

Epoch 00311: val_loss did not improve from 0.66956
Epoch 312/500
 - 3s - loss: 0.6250 - acc: 0.8069 - val_loss: 0.6858 - val_acc: 0.7746

Epoch 00312: val_loss did not improve from 0.66956
Epoch 313/500
 - 3s - loss: 0.6269 - acc: 0.8054 - val_loss: 0.7099 - val_acc: 0.7734

Epoch 00313: val_loss did not improve from 0.66956
Epoch 314/500
 - 3s - loss: 0.6353 - acc: 0.8024 - val_loss: 0.6855 - val_acc: 0.7742

Epoch 00314: val_loss did not improve from 0.66956
Epoch 315/500
 - 3s - loss: 0.6209 - acc: 0.8097 - val_loss: 0.6911 - val_acc: 0.7805

Epoch 00315: val_loss did not improve from 0.66956
Epoch 316/500
 - 3s - loss: 0.6224 - acc: 0.8091 - val_loss: 0.7003 - val_acc: 0.7658

Epoch 00316: val_loss did not improve from 0.66956
Epoch 317/500
 - 3s - loss: 0.6378 - acc: 0.7986 - val_loss: 0.6826 - val_acc: 0.7824

Epoch 00317: val_loss did not improve from 0.66956
Epoch 318/500
 - 3s - loss: 0.6156 - acc: 0.8147 - val_loss: 0.6748 - val_acc: 0.7833

Epoch 00318: val_loss did not improve from 0.66956
Epoch 319/500
 - 3s - loss: 0.6186 - acc: 0.8090 - val_loss: 0.7245 - val_acc: 0.7701

Epoch 00319: val_loss did not improve from 0.66956
Epoch 320/500
 - 3s - loss: 0.6412 - acc: 0.7977 - val_loss: 0.6718 - val_acc: 0.7828

Epoch 00320: val_loss did not improve from 0.66956
Epoch 321/500
 - 3s - loss: 0.6133 - acc: 0.8136 - val_loss: 0.6947 - val_acc: 0.7858

Epoch 00321: val_loss did not improve from 0.66956
Epoch 322/500
 - 3s - loss: 0.6242 - acc: 0.8077 - val_loss: 0.6928 - val_acc: 0.7716

Epoch 00322: val_loss did not improve from 0.66956
Epoch 323/500
 - 3s - loss: 0.6327 - acc: 0.8021 - val_loss: 0.6760 - val_acc: 0.7788

Epoch 00323: val_loss did not improve from 0.66956
Epoch 324/500
 - 3s - loss: 0.6144 - acc: 0.8145 - val_loss: 0.7220 - val_acc: 0.7778

Epoch 00324: val_loss did not improve from 0.66956
Epoch 325/500
 - 3s - loss: 0.6310 - acc: 0.8048 - val_loss: 0.6808 - val_acc: 0.7741

Epoch 00325: val_loss did not improve from 0.66956
Epoch 326/500
 - 3s - loss: 0.6218 - acc: 0.8074 - val_loss: 0.7095 - val_acc: 0.7757

Epoch 00326: val_loss did not improve from 0.66956
Epoch 327/500
 - 3s - loss: 0.6301 - acc: 0.8044 - val_loss: 0.6728 - val_acc: 0.7814

Epoch 00327: val_loss did not improve from 0.66956
Epoch 328/500
 - 3s - loss: 0.6147 - acc: 0.8122 - val_loss: 0.6847 - val_acc: 0.7829

Epoch 00328: val_loss did not improve from 0.66956
Epoch 329/500
 - 3s - loss: 0.6254 - acc: 0.8091 - val_loss: 0.6806 - val_acc: 0.7738

Epoch 00329: val_loss did not improve from 0.66956
Epoch 330/500
 - 3s - loss: 0.6232 - acc: 0.8080 - val_loss: 0.7037 - val_acc: 0.7768

Epoch 00330: val_loss did not improve from 0.66956
Epoch 331/500
 - 3s - loss: 0.6248 - acc: 0.8074 - val_loss: 0.7197 - val_acc: 0.7563

Epoch 00331: val_loss did not improve from 0.66956
Epoch 332/500
 - 3s - loss: 0.6277 - acc: 0.8043 - val_loss: 0.6817 - val_acc: 0.7796

Epoch 00332: val_loss did not improve from 0.66956
Epoch 333/500
 - 3s - loss: 0.6147 - acc: 0.8154 - val_loss: 0.6929 - val_acc: 0.7656

Epoch 00333: val_loss did not improve from 0.66956
Epoch 334/500
 - 3s - loss: 0.6359 - acc: 0.7978 - val_loss: 0.6888 - val_acc: 0.7804

Epoch 00334: val_loss did not improve from 0.66956
Epoch 335/500
 - 3s - loss: 0.6177 - acc: 0.8105 - val_loss: 0.6795 - val_acc: 0.7782

Epoch 00335: val_loss did not improve from 0.66956
Epoch 336/500
 - 3s - loss: 0.6127 - acc: 0.8142 - val_loss: 0.6866 - val_acc: 0.7779

Epoch 00336: val_loss did not improve from 0.66956
Epoch 337/500
 - 3s - loss: 0.6114 - acc: 0.8154 - val_loss: 0.6868 - val_acc: 0.7788

Epoch 00337: val_loss did not improve from 0.66956
Epoch 338/500
 - 3s - loss: 0.6335 - acc: 0.8016 - val_loss: 0.6754 - val_acc: 0.7840

Epoch 00338: val_loss did not improve from 0.66956
Epoch 339/500
 - 3s - loss: 0.6112 - acc: 0.8151 - val_loss: 0.7180 - val_acc: 0.7721

Epoch 00339: val_loss did not improve from 0.66956
Epoch 340/500
 - 3s - loss: 0.6313 - acc: 0.8028 - val_loss: 0.6779 - val_acc: 0.7786

Epoch 00340: val_loss did not improve from 0.66956
Epoch 341/500
 - 3s - loss: 0.6113 - acc: 0.8116 - val_loss: 0.6905 - val_acc: 0.7799

Epoch 00341: val_loss did not improve from 0.66956
Epoch 342/500
 - 3s - loss: 0.6190 - acc: 0.8088 - val_loss: 0.7001 - val_acc: 0.7614

Epoch 00342: val_loss did not improve from 0.66956
Epoch 343/500
 - 3s - loss: 0.6224 - acc: 0.8069 - val_loss: 0.6720 - val_acc: 0.7876

Epoch 00343: val_loss did not improve from 0.66956
Epoch 344/500
 - 3s - loss: 0.6091 - acc: 0.8159 - val_loss: 0.6828 - val_acc: 0.7814

Epoch 00344: val_loss did not improve from 0.66956
Epoch 345/500
 - 3s - loss: 0.6228 - acc: 0.8077 - val_loss: 0.6997 - val_acc: 0.7592

Epoch 00345: val_loss did not improve from 0.66956
Epoch 346/500
 - 3s - loss: 0.6226 - acc: 0.8068 - val_loss: 0.6754 - val_acc: 0.7841

Epoch 00346: val_loss did not improve from 0.66956
Epoch 347/500
 - 3s - loss: 0.6079 - acc: 0.8148 - val_loss: 0.6992 - val_acc: 0.7683

Epoch 00347: val_loss did not improve from 0.66956
Epoch 348/500
 - 3s - loss: 0.6383 - acc: 0.7973 - val_loss: 0.6826 - val_acc: 0.7844

Epoch 00348: val_loss did not improve from 0.66956
Epoch 349/500
 - 3s - loss: 0.6099 - acc: 0.8151 - val_loss: 0.6729 - val_acc: 0.7849

Epoch 00349: val_loss did not improve from 0.66956
Epoch 350/500
 - 3s - loss: 0.6285 - acc: 0.8038 - val_loss: 0.6803 - val_acc: 0.7835

Epoch 00350: val_loss did not improve from 0.66956
Epoch 351/500
 - 3s - loss: 0.6067 - acc: 0.8172 - val_loss: 0.6963 - val_acc: 0.7813

Epoch 00351: val_loss did not improve from 0.66956
Epoch 352/500
 - 3s - loss: 0.6166 - acc: 0.8138 - val_loss: 0.6897 - val_acc: 0.7740

Epoch 00352: val_loss did not improve from 0.66956
Epoch 353/500
 - 4s - loss: 0.6420 - acc: 0.7946 - val_loss: 0.6964 - val_acc: 0.7768

Epoch 00353: val_loss did not improve from 0.66956
Epoch 354/500
 - 3s - loss: 0.6182 - acc: 0.8095 - val_loss: 0.6828 - val_acc: 0.7742

Epoch 00354: val_loss did not improve from 0.66956
Epoch 355/500
 - 4s - loss: 0.6180 - acc: 0.8090 - val_loss: 0.6728 - val_acc: 0.7876

Epoch 00355: val_loss did not improve from 0.66956
Epoch 356/500
 - 3s - loss: 0.6123 - acc: 0.8141 - val_loss: 0.7266 - val_acc: 0.7536

Epoch 00356: val_loss did not improve from 0.66956
Epoch 357/500
 - 3s - loss: 0.6327 - acc: 0.8016 - val_loss: 0.6785 - val_acc: 0.7823

Epoch 00357: val_loss did not improve from 0.66956
Epoch 358/500
 - 3s - loss: 0.6072 - acc: 0.8159 - val_loss: 0.6885 - val_acc: 0.7806

Epoch 00358: val_loss did not improve from 0.66956
Epoch 359/500
 - 3s - loss: 0.6229 - acc: 0.8082 - val_loss: 0.6938 - val_acc: 0.7653

Epoch 00359: val_loss did not improve from 0.66956
Epoch 360/500
 - 3s - loss: 0.6205 - acc: 0.8073 - val_loss: 0.6875 - val_acc: 0.7829

Epoch 00360: val_loss did not improve from 0.66956
Epoch 361/500
 - 3s - loss: 0.6166 - acc: 0.8129 - val_loss: 0.6966 - val_acc: 0.7735

Epoch 00361: val_loss did not improve from 0.66956
Epoch 362/500
 - 3s - loss: 0.6244 - acc: 0.8065 - val_loss: 0.6856 - val_acc: 0.7776

Epoch 00362: val_loss did not improve from 0.66956
Epoch 363/500
 - 3s - loss: 0.6166 - acc: 0.8098 - val_loss: 0.7199 - val_acc: 0.7512

Epoch 00363: val_loss did not improve from 0.66956
Epoch 364/500
 - 3s - loss: 0.6274 - acc: 0.8040 - val_loss: 0.6790 - val_acc: 0.7870

Epoch 00364: val_loss did not improve from 0.66956
Epoch 365/500
 - 3s - loss: 0.6091 - acc: 0.8160 - val_loss: 0.6744 - val_acc: 0.7788

Epoch 00365: val_loss did not improve from 0.66956
Epoch 366/500
 - 3s - loss: 0.6229 - acc: 0.8059 - val_loss: 0.6974 - val_acc: 0.7805

Epoch 00366: val_loss did not improve from 0.66956
Epoch 367/500
 - 3s - loss: 0.6139 - acc: 0.8123 - val_loss: 0.6802 - val_acc: 0.7837

Epoch 00367: val_loss did not improve from 0.66956
Epoch 368/500
 - 3s - loss: 0.6170 - acc: 0.8109 - val_loss: 0.6954 - val_acc: 0.7827

Epoch 00368: val_loss did not improve from 0.66956
Epoch 369/500
 - 3s - loss: 0.6224 - acc: 0.8082 - val_loss: 0.6846 - val_acc: 0.7704

Epoch 00369: val_loss did not improve from 0.66956
Epoch 370/500
 - 3s - loss: 0.6184 - acc: 0.8098 - val_loss: 0.6834 - val_acc: 0.7845

Epoch 00370: val_loss did not improve from 0.66956
Epoch 371/500
 - 3s - loss: 0.6148 - acc: 0.8127 - val_loss: 0.7027 - val_acc: 0.7627

Epoch 00371: val_loss did not improve from 0.66956
Epoch 372/500
 - 3s - loss: 0.6243 - acc: 0.8090 - val_loss: 0.6754 - val_acc: 0.7879

Epoch 00372: val_loss did not improve from 0.66956
Epoch 373/500
 - 3s - loss: 0.6046 - acc: 0.8178 - val_loss: 0.6973 - val_acc: 0.7770

Epoch 00373: val_loss did not improve from 0.66956
Epoch 374/500
 - 3s - loss: 0.6325 - acc: 0.8012 - val_loss: 0.6921 - val_acc: 0.7693

Epoch 00374: val_loss did not improve from 0.66956
Epoch 375/500
 - 3s - loss: 0.6140 - acc: 0.8131 - val_loss: 0.6838 - val_acc: 0.7821

Epoch 00375: val_loss did not improve from 0.66956
Epoch 376/500
 - 3s - loss: 0.6114 - acc: 0.8143 - val_loss: 0.6828 - val_acc: 0.7851

Epoch 00376: val_loss did not improve from 0.66956
Epoch 377/500
 - 3s - loss: 0.6132 - acc: 0.8131 - val_loss: 0.6777 - val_acc: 0.7825

Epoch 00377: val_loss did not improve from 0.66956
Epoch 378/500
 - 3s - loss: 0.6131 - acc: 0.8141 - val_loss: 0.6975 - val_acc: 0.7768

Epoch 00378: val_loss did not improve from 0.66956
Epoch 379/500
 - 3s - loss: 0.6225 - acc: 0.8066 - val_loss: 0.6951 - val_acc: 0.7701

Epoch 00379: val_loss did not improve from 0.66956
Epoch 380/500
 - 3s - loss: 0.6195 - acc: 0.8084 - val_loss: 0.6755 - val_acc: 0.7841

Epoch 00380: val_loss did not improve from 0.66956
Epoch 381/500
 - 3s - loss: 0.6061 - acc: 0.8181 - val_loss: 0.6804 - val_acc: 0.7770

Epoch 00381: val_loss did not improve from 0.66956
Epoch 382/500
 - 3s - loss: 0.6165 - acc: 0.8097 - val_loss: 0.7139 - val_acc: 0.7728

Epoch 00382: val_loss did not improve from 0.66956
Epoch 383/500
 - 3s - loss: 0.6263 - acc: 0.8045 - val_loss: 0.6773 - val_acc: 0.7839

Epoch 00383: val_loss did not improve from 0.66956
Epoch 384/500
 - 3s - loss: 0.6040 - acc: 0.8186 - val_loss: 0.7015 - val_acc: 0.7698

Epoch 00384: val_loss did not improve from 0.66956
Epoch 385/500
 - 3s - loss: 0.6329 - acc: 0.7998 - val_loss: 0.6737 - val_acc: 0.7894

Epoch 00385: val_loss did not improve from 0.66956
Epoch 386/500
 - 3s - loss: 0.6080 - acc: 0.8154 - val_loss: 0.6927 - val_acc: 0.7717

Epoch 00386: val_loss did not improve from 0.66956
Epoch 387/500
 - 3s - loss: 0.6238 - acc: 0.8060 - val_loss: 0.6859 - val_acc: 0.7833

Epoch 00387: val_loss did not improve from 0.66956
Epoch 388/500
 - 3s - loss: 0.6087 - acc: 0.8138 - val_loss: 0.6822 - val_acc: 0.7742

Epoch 00388: val_loss did not improve from 0.66956
Epoch 389/500
 - 3s - loss: 0.6163 - acc: 0.8089 - val_loss: 0.6913 - val_acc: 0.7811

Epoch 00389: val_loss did not improve from 0.66956
Epoch 390/500
 - 3s - loss: 0.6118 - acc: 0.8122 - val_loss: 0.6939 - val_acc: 0.7713

Epoch 00390: val_loss did not improve from 0.66956
Epoch 391/500
 - 3s - loss: 0.6272 - acc: 0.8045 - val_loss: 0.6903 - val_acc: 0.7816

Epoch 00391: val_loss did not improve from 0.66956
Epoch 392/500
 - 3s - loss: 0.6090 - acc: 0.8147 - val_loss: 0.6843 - val_acc: 0.7783

Epoch 00392: val_loss did not improve from 0.66956
Epoch 393/500
 - 3s - loss: 0.6049 - acc: 0.8159 - val_loss: 0.6827 - val_acc: 0.7807

Epoch 00393: val_loss did not improve from 0.66956
Epoch 394/500
 - 3s - loss: 0.6230 - acc: 0.8072 - val_loss: 0.6986 - val_acc: 0.7671

Epoch 00394: val_loss did not improve from 0.66956
Epoch 395/500
 - 3s - loss: 0.6164 - acc: 0.8116 - val_loss: 0.6831 - val_acc: 0.7869

Epoch 00395: val_loss did not improve from 0.66956
Epoch 396/500
 - 3s - loss: 0.6080 - acc: 0.8172 - val_loss: 0.6780 - val_acc: 0.7836

Epoch 00396: val_loss did not improve from 0.66956
Epoch 397/500
 - 3s - loss: 0.6033 - acc: 0.8174 - val_loss: 0.6927 - val_acc: 0.7654

Epoch 00397: val_loss did not improve from 0.66956
Epoch 398/500
 - 3s - loss: 0.6430 - acc: 0.7956 - val_loss: 0.6853 - val_acc: 0.7859

Epoch 00398: val_loss did not improve from 0.66956
Epoch 399/500
 - 3s - loss: 0.6064 - acc: 0.8174 - val_loss: 0.6876 - val_acc: 0.7723

Epoch 00399: val_loss did not improve from 0.66956
Epoch 400/500
 - 3s - loss: 0.6186 - acc: 0.8060 - val_loss: 0.6945 - val_acc: 0.7809

Epoch 00400: val_loss did not improve from 0.66956
Epoch 401/500
 - 3s - loss: 0.6148 - acc: 0.8127 - val_loss: 0.6870 - val_acc: 0.7762

Epoch 00401: val_loss did not improve from 0.66956
Epoch 402/500
 - 3s - loss: 0.6156 - acc: 0.8098 - val_loss: 0.7292 - val_acc: 0.7708

Epoch 00402: val_loss did not improve from 0.66956
Epoch 403/500
 - 3s - loss: 0.6235 - acc: 0.8077 - val_loss: 0.6800 - val_acc: 0.7790

Epoch 00403: val_loss did not improve from 0.66956
Epoch 404/500
 - 3s - loss: 0.6034 - acc: 0.8200 - val_loss: 0.6870 - val_acc: 0.7801

Epoch 00404: val_loss did not improve from 0.66956
Epoch 405/500
 - 3s - loss: 0.6031 - acc: 0.8191 - val_loss: 0.6777 - val_acc: 0.7821

Epoch 00405: val_loss did not improve from 0.66956
Epoch 406/500
 - 3s - loss: 0.6048 - acc: 0.8169 - val_loss: 0.7706 - val_acc: 0.7231

Epoch 00406: val_loss did not improve from 0.66956
Epoch 407/500
 - 3s - loss: 0.6424 - acc: 0.7955 - val_loss: 0.6808 - val_acc: 0.7852

Epoch 00407: val_loss did not improve from 0.66956
Epoch 408/500
 - 3s - loss: 0.6002 - acc: 0.8205 - val_loss: 0.6868 - val_acc: 0.7815

Epoch 00408: val_loss did not improve from 0.66956
Epoch 409/500
 - 3s - loss: 0.6067 - acc: 0.8182 - val_loss: 0.7265 - val_acc: 0.7481

Epoch 00409: val_loss did not improve from 0.66956
Epoch 410/500
 - 3s - loss: 0.6344 - acc: 0.8003 - val_loss: 0.6797 - val_acc: 0.7799

Epoch 00410: val_loss did not improve from 0.66956
Epoch 411/500
 - 3s - loss: 0.6025 - acc: 0.8187 - val_loss: 0.7259 - val_acc: 0.7490

Epoch 00411: val_loss did not improve from 0.66956
Epoch 412/500
 - 3s - loss: 0.6306 - acc: 0.8006 - val_loss: 0.6872 - val_acc: 0.7815

Epoch 00412: val_loss did not improve from 0.66956
Epoch 413/500
 - 3s - loss: 0.6048 - acc: 0.8174 - val_loss: 0.6899 - val_acc: 0.7802

Epoch 00413: val_loss did not improve from 0.66956
Epoch 414/500
 - 3s - loss: 0.6163 - acc: 0.8097 - val_loss: 0.6870 - val_acc: 0.7822

Epoch 00414: val_loss did not improve from 0.66956
Epoch 415/500
 - 3s - loss: 0.6119 - acc: 0.8120 - val_loss: 0.6676 - val_acc: 0.7872

Epoch 00415: val_loss improved from 0.66956 to 0.66761, saving model to weights-improvement-best-model.hdf5
Epoch 416/500
 - 3s - loss: 0.6052 - acc: 0.8171 - val_loss: 0.6906 - val_acc: 0.7774

Epoch 00416: val_loss did not improve from 0.66761
Epoch 417/500
 - 3s - loss: 0.6135 - acc: 0.8133 - val_loss: 0.7054 - val_acc: 0.7653

Epoch 00417: val_loss did not improve from 0.66761
Epoch 418/500
 - 3s - loss: 0.6190 - acc: 0.8096 - val_loss: 0.6728 - val_acc: 0.7849

Epoch 00418: val_loss did not improve from 0.66761
Epoch 419/500
 - 3s - loss: 0.6028 - acc: 0.8203 - val_loss: 0.7077 - val_acc: 0.7666

Epoch 00419: val_loss did not improve from 0.66761
Epoch 420/500
 - 3s - loss: 0.6248 - acc: 0.8042 - val_loss: 0.7219 - val_acc: 0.7775

Epoch 00420: val_loss did not improve from 0.66761
Epoch 421/500
 - 3s - loss: 0.6206 - acc: 0.8118 - val_loss: 0.6797 - val_acc: 0.7825

Epoch 00421: val_loss did not improve from 0.66761
Epoch 422/500
 - 3s - loss: 0.6019 - acc: 0.8189 - val_loss: 0.6961 - val_acc: 0.7827

Epoch 00422: val_loss did not improve from 0.66761
Epoch 423/500
 - 3s - loss: 0.6197 - acc: 0.8097 - val_loss: 0.7074 - val_acc: 0.7636

Epoch 00423: val_loss did not improve from 0.66761
Epoch 424/500
 - 3s - loss: 0.6163 - acc: 0.8114 - val_loss: 0.6816 - val_acc: 0.7768

Epoch 00424: val_loss did not improve from 0.66761
Epoch 425/500
 - 3s - loss: 0.6037 - acc: 0.8181 - val_loss: 0.6910 - val_acc: 0.7820

Epoch 00425: val_loss did not improve from 0.66761
Epoch 426/500
 - 3s - loss: 0.6169 - acc: 0.8111 - val_loss: 0.6842 - val_acc: 0.7723

Epoch 00426: val_loss did not improve from 0.66761
Epoch 427/500
 - 3s - loss: 0.6109 - acc: 0.8139 - val_loss: 0.6749 - val_acc: 0.7792

Epoch 00427: val_loss did not improve from 0.66761
Epoch 428/500
 - 3s - loss: 0.6024 - acc: 0.8217 - val_loss: 0.6964 - val_acc: 0.7743

Epoch 00428: val_loss did not improve from 0.66761
Epoch 429/500
 - 3s - loss: 0.6190 - acc: 0.8088 - val_loss: 0.6916 - val_acc: 0.7692

Epoch 00429: val_loss did not improve from 0.66761
Epoch 430/500
 - 3s - loss: 0.6116 - acc: 0.8139 - val_loss: 0.6800 - val_acc: 0.7852

Epoch 00430: val_loss did not improve from 0.66761
Epoch 431/500
 - 3s - loss: 0.6043 - acc: 0.8171 - val_loss: 0.7123 - val_acc: 0.7513

Epoch 00431: val_loss did not improve from 0.66761
Epoch 432/500
 - 3s - loss: 0.6180 - acc: 0.8081 - val_loss: 0.6829 - val_acc: 0.7790

Epoch 00432: val_loss did not improve from 0.66761
Epoch 433/500
 - 3s - loss: 0.6013 - acc: 0.8210 - val_loss: 0.6839 - val_acc: 0.7852

Epoch 00433: val_loss did not improve from 0.66761
Epoch 434/500
 - 3s - loss: 0.6114 - acc: 0.8132 - val_loss: 0.7026 - val_acc: 0.7693

Epoch 00434: val_loss did not improve from 0.66761
Epoch 435/500
 - 3s - loss: 0.6097 - acc: 0.8133 - val_loss: 0.6997 - val_acc: 0.7800

Epoch 00435: val_loss did not improve from 0.66761
Epoch 436/500
 - 3s - loss: 0.6162 - acc: 0.8111 - val_loss: 0.7092 - val_acc: 0.7681

Epoch 00436: val_loss did not improve from 0.66761
Epoch 437/500
 - 3s - loss: 0.6096 - acc: 0.8133 - val_loss: 0.6809 - val_acc: 0.7842

Epoch 00437: val_loss did not improve from 0.66761
Epoch 438/500
 - 3s - loss: 0.6027 - acc: 0.8174 - val_loss: 0.6960 - val_acc: 0.7666

Epoch 00438: val_loss did not improve from 0.66761
Epoch 439/500
 - 3s - loss: 0.6122 - acc: 0.8108 - val_loss: 0.6912 - val_acc: 0.7704

Epoch 00439: val_loss did not improve from 0.66761
Epoch 440/500
 - 3s - loss: 0.6073 - acc: 0.8178 - val_loss: 0.6849 - val_acc: 0.7848

Epoch 00440: val_loss did not improve from 0.66761
Epoch 441/500
 - 3s - loss: 0.6105 - acc: 0.8125 - val_loss: 0.7076 - val_acc: 0.7648

Epoch 00441: val_loss did not improve from 0.66761
Epoch 442/500
 - 3s - loss: 0.6132 - acc: 0.8105 - val_loss: 0.6832 - val_acc: 0.7847

Epoch 00442: val_loss did not improve from 0.66761
Epoch 443/500
 - 3s - loss: 0.5998 - acc: 0.8207 - val_loss: 0.7214 - val_acc: 0.7524

Epoch 00443: val_loss did not improve from 0.66761
Epoch 444/500
 - 3s - loss: 0.6242 - acc: 0.8046 - val_loss: 0.6968 - val_acc: 0.7763

Epoch 00444: val_loss did not improve from 0.66761
Epoch 445/500
 - 3s - loss: 0.6127 - acc: 0.8138 - val_loss: 0.7058 - val_acc: 0.7775

Epoch 00445: val_loss did not improve from 0.66761
Epoch 446/500
 - 3s - loss: 0.6074 - acc: 0.8164 - val_loss: 0.6861 - val_acc: 0.7738

Epoch 00446: val_loss did not improve from 0.66761
Epoch 447/500
 - 3s - loss: 0.6011 - acc: 0.8219 - val_loss: 0.6864 - val_acc: 0.7830

Epoch 00447: val_loss did not improve from 0.66761
Epoch 448/500
 - 3s - loss: 0.6039 - acc: 0.8171 - val_loss: 0.6899 - val_acc: 0.7759

Epoch 00448: val_loss did not improve from 0.66761
Epoch 449/500
 - 3s - loss: 0.6087 - acc: 0.8153 - val_loss: 0.6948 - val_acc: 0.7800

Epoch 00449: val_loss did not improve from 0.66761
Epoch 450/500
 - 3s - loss: 0.6058 - acc: 0.8158 - val_loss: 0.6865 - val_acc: 0.7755

Epoch 00450: val_loss did not improve from 0.66761
Epoch 451/500
 - 3s - loss: 0.6015 - acc: 0.8190 - val_loss: 0.6933 - val_acc: 0.7814

Epoch 00451: val_loss did not improve from 0.66761
Epoch 452/500
 - 3s - loss: 0.6162 - acc: 0.8122 - val_loss: 0.6995 - val_acc: 0.7672

Epoch 00452: val_loss did not improve from 0.66761
Epoch 453/500
 - 3s - loss: 0.6134 - acc: 0.8145 - val_loss: 0.6786 - val_acc: 0.7795

Epoch 00453: val_loss did not improve from 0.66761
Epoch 454/500
 - 4s - loss: 0.5942 - acc: 0.8243 - val_loss: 0.6872 - val_acc: 0.7845

Epoch 00454: val_loss did not improve from 0.66761
Epoch 455/500
 - 4s - loss: 0.6156 - acc: 0.8126 - val_loss: 0.7103 - val_acc: 0.7786

Epoch 00455: val_loss did not improve from 0.66761
Epoch 456/500
 - 4s - loss: 0.6165 - acc: 0.8103 - val_loss: 0.6899 - val_acc: 0.7694

Epoch 00456: val_loss did not improve from 0.66761
Epoch 457/500
 - 4s - loss: 0.6024 - acc: 0.8182 - val_loss: 0.6872 - val_acc: 0.7815

Epoch 00457: val_loss did not improve from 0.66761
Epoch 458/500
 - 4s - loss: 0.6082 - acc: 0.8163 - val_loss: 0.7075 - val_acc: 0.7644

Epoch 00458: val_loss did not improve from 0.66761
Epoch 459/500
 - 3s - loss: 0.6153 - acc: 0.8091 - val_loss: 0.6998 - val_acc: 0.7818

Epoch 00459: val_loss did not improve from 0.66761
Epoch 460/500
 - 3s - loss: 0.6000 - acc: 0.8214 - val_loss: 0.6965 - val_acc: 0.7636

Epoch 00460: val_loss did not improve from 0.66761
Epoch 461/500
 - 3s - loss: 0.6201 - acc: 0.8071 - val_loss: 0.7067 - val_acc: 0.7867

Epoch 00461: val_loss did not improve from 0.66761
Epoch 462/500
 - 3s - loss: 0.6019 - acc: 0.8185 - val_loss: 0.7207 - val_acc: 0.7739

Epoch 00462: val_loss did not improve from 0.66761
Epoch 463/500
 - 3s - loss: 0.6146 - acc: 0.8113 - val_loss: 0.7035 - val_acc: 0.7811

Epoch 00463: val_loss did not improve from 0.66761
Epoch 464/500
 - 3s - loss: 0.6071 - acc: 0.8155 - val_loss: 0.7011 - val_acc: 0.7678

Epoch 00464: val_loss did not improve from 0.66761
Epoch 465/500
 - 3s - loss: 0.6172 - acc: 0.8087 - val_loss: 0.6810 - val_acc: 0.7828

Epoch 00465: val_loss did not improve from 0.66761
Epoch 466/500
 - 3s - loss: 0.5972 - acc: 0.8215 - val_loss: 0.6888 - val_acc: 0.7730

Epoch 00466: val_loss did not improve from 0.66761
Epoch 467/500
 - 3s - loss: 0.6108 - acc: 0.8107 - val_loss: 0.6896 - val_acc: 0.7798

Epoch 00467: val_loss did not improve from 0.66761
Epoch 468/500
 - 3s - loss: 0.5997 - acc: 0.8195 - val_loss: 0.7396 - val_acc: 0.7479

Epoch 00468: val_loss did not improve from 0.66761
Epoch 469/500
 - 3s - loss: 0.6247 - acc: 0.8044 - val_loss: 0.6798 - val_acc: 0.7826

Epoch 00469: val_loss did not improve from 0.66761
Epoch 470/500
 - 3s - loss: 0.5963 - acc: 0.8209 - val_loss: 0.6866 - val_acc: 0.7768

Epoch 00470: val_loss did not improve from 0.66761
Epoch 471/500
 - 3s - loss: 0.6041 - acc: 0.8186 - val_loss: 0.7183 - val_acc: 0.7756

Epoch 00471: val_loss did not improve from 0.66761
Epoch 472/500
 - 3s - loss: 0.6138 - acc: 0.8122 - val_loss: 0.6831 - val_acc: 0.7741

Epoch 00472: val_loss did not improve from 0.66761
Epoch 473/500
 - 3s - loss: 0.5992 - acc: 0.8213 - val_loss: 0.7156 - val_acc: 0.7745

Epoch 00473: val_loss did not improve from 0.66761
Epoch 474/500
 - 3s - loss: 0.6215 - acc: 0.8081 - val_loss: 0.7073 - val_acc: 0.7764

Epoch 00474: val_loss did not improve from 0.66761
Epoch 475/500
 - 3s - loss: 0.6057 - acc: 0.8185 - val_loss: 0.6968 - val_acc: 0.7795

Epoch 00475: val_loss did not improve from 0.66761
Epoch 476/500
 - 3s - loss: 0.5988 - acc: 0.8199 - val_loss: 0.6983 - val_acc: 0.7779

Epoch 00476: val_loss did not improve from 0.66761
Epoch 477/500
 - 3s - loss: 0.5973 - acc: 0.8208 - val_loss: 0.6854 - val_acc: 0.7783

Epoch 00477: val_loss did not improve from 0.66761
Epoch 478/500
 - 3s - loss: 0.6129 - acc: 0.8105 - val_loss: 0.7018 - val_acc: 0.7629

Epoch 00478: val_loss did not improve from 0.66761
Epoch 479/500
 - 3s - loss: 0.6070 - acc: 0.8166 - val_loss: 0.6746 - val_acc: 0.7796

Epoch 00479: val_loss did not improve from 0.66761
Epoch 480/500
 - 3s - loss: 0.5924 - acc: 0.8253 - val_loss: 0.7259 - val_acc: 0.7592

Epoch 00480: val_loss did not improve from 0.66761
Epoch 481/500
 - 3s - loss: 0.6275 - acc: 0.8022 - val_loss: 0.6822 - val_acc: 0.7780

Epoch 00481: val_loss did not improve from 0.66761
Epoch 482/500
 - 3s - loss: 0.5975 - acc: 0.8194 - val_loss: 0.6873 - val_acc: 0.7817

Epoch 00482: val_loss did not improve from 0.66761
Epoch 483/500
 - 3s - loss: 0.6011 - acc: 0.8199 - val_loss: 0.6959 - val_acc: 0.7696

Epoch 00483: val_loss did not improve from 0.66761
Epoch 484/500
 - 3s - loss: 0.6074 - acc: 0.8169 - val_loss: 0.7176 - val_acc: 0.7721

Epoch 00484: val_loss did not improve from 0.66761
Epoch 485/500
 - 3s - loss: 0.6173 - acc: 0.8103 - val_loss: 0.6835 - val_acc: 0.7828

Epoch 00485: val_loss did not improve from 0.66761
Epoch 486/500
 - 3s - loss: 0.5919 - acc: 0.8239 - val_loss: 0.7227 - val_acc: 0.7484

Epoch 00486: val_loss did not improve from 0.66761
Epoch 487/500
 - 3s - loss: 0.6256 - acc: 0.8012 - val_loss: 0.6888 - val_acc: 0.7819

Epoch 00487: val_loss did not improve from 0.66761
Epoch 488/500
 - 3s - loss: 0.6009 - acc: 0.8198 - val_loss: 0.6865 - val_acc: 0.7831

Epoch 00488: val_loss did not improve from 0.66761
Epoch 489/500
 - 3s - loss: 0.5929 - acc: 0.8218 - val_loss: 0.6826 - val_acc: 0.7771

Epoch 00489: val_loss did not improve from 0.66761
Epoch 490/500
 - 3s - loss: 0.6015 - acc: 0.8201 - val_loss: 0.7042 - val_acc: 0.7746

Epoch 00490: val_loss did not improve from 0.66761
Epoch 491/500
 - 3s - loss: 0.6278 - acc: 0.8021 - val_loss: 0.6886 - val_acc: 0.7717

Epoch 00491: val_loss did not improve from 0.66761
Epoch 492/500
 - 3s - loss: 0.5970 - acc: 0.8213 - val_loss: 0.6987 - val_acc: 0.7821

Epoch 00492: val_loss did not improve from 0.66761
Epoch 493/500
 - 3s - loss: 0.6124 - acc: 0.8130 - val_loss: 0.7058 - val_acc: 0.7672

Epoch 00493: val_loss did not improve from 0.66761
Epoch 494/500
 - 3s - loss: 0.6058 - acc: 0.8143 - val_loss: 0.6868 - val_acc: 0.7867

Epoch 00494: val_loss did not improve from 0.66761
Epoch 495/500
 - 3s - loss: 0.5987 - acc: 0.8198 - val_loss: 0.7000 - val_acc: 0.7734

Epoch 00495: val_loss did not improve from 0.66761
Epoch 496/500
 - 3s - loss: 0.6132 - acc: 0.8113 - val_loss: 0.7104 - val_acc: 0.7779

Epoch 00496: val_loss did not improve from 0.66761
Epoch 497/500
 - 3s - loss: 0.6042 - acc: 0.8166 - val_loss: 0.6733 - val_acc: 0.7845

Epoch 00497: val_loss did not improve from 0.66761
Epoch 498/500
 - 3s - loss: 0.5922 - acc: 0.8248 - val_loss: 0.6995 - val_acc: 0.7842

Epoch 00498: val_loss did not improve from 0.66761
Epoch 499/500
 - 3s - loss: 0.6194 - acc: 0.8080 - val_loss: 0.6885 - val_acc: 0.7747

Epoch 00499: val_loss did not improve from 0.66761
Epoch 500/500
 - 3s - loss: 0.6000 - acc: 0.8198 - val_loss: 0.7014 - val_acc: 0.7760

Epoch 00500: val_loss did not improve from 0.66761

# trying another architechture : model_1
model_1 = Sequential()
model_1.add(Dense(1024, kernel_regularizer=regularizers.l2(.001),input_dim=X_train.shape[1]))
model_1.add(Activation('relu'))
model_1.add(Dropout(0.2))
model_1.add(Dense(512,kernel_regularizer=regularizers.l2(.001)))
model_1.add(Activation('relu'))
model_1.add(Dropout(0.2))
model_1.add(Dense(512,kernel_regularizer=regularizers.l2(.001)))
model_1.add(Activation('relu'))
model_1.add(Dropout(0.2))
model_1.add(Dense(128,kernel_regularizer=regularizers.l1_l2(.001)))
model_1.add(Activation('relu'))
model_1.add(Dropout(0.3))
model_1.add(Dense(32, kernel_regularizer=regularizers.l2(.001)))
model_1.add(Activation('relu'))
model_1.add(Dense(3, activation = 'softmax'))

model_1.summary()

_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
dense_47 (Dense)             (None, 1024)              140288    
_________________________________________________________________
activation_37 (Activation)   (None, 1024)              0         
_________________________________________________________________
dropout_28 (Dropout)         (None, 1024)              0         
_________________________________________________________________
dense_48 (Dense)             (None, 512)               524800    
_________________________________________________________________
activation_38 (Activation)   (None, 512)               0         
_________________________________________________________________
dropout_29 (Dropout)         (None, 512)               0         
_________________________________________________________________
dense_49 (Dense)             (None, 512)               262656    
_________________________________________________________________
activation_39 (Activation)   (None, 512)               0         
_________________________________________________________________
dropout_30 (Dropout)         (None, 512)               0         
_________________________________________________________________
dense_50 (Dense)             (None, 128)               65664     
_________________________________________________________________
activation_40 (Activation)   (None, 128)               0         
_________________________________________________________________
dropout_31 (Dropout)         (None, 128)               0         
_________________________________________________________________
dense_51 (Dense)             (None, 32)                4128      
_________________________________________________________________
activation_41 (Activation)   (None, 32)                0         
_________________________________________________________________
dense_52 (Dense)             (None, 3)                 99        
=================================================================
Total params: 997,635
Trainable params: 997,635
Non-trainable params: 0
_________________________________________________________________

# compile the model_1
model_1.compile(optimizer='rmsprop', loss='categorical_crossentropy', metrics=['accuracy'])

# checkpoint
filepath="weights-improvement-best-model_1.hdf5"
checkpoint = ModelCheckpoint(filepath, monitor='val_loss', verbose=1, save_best_only=True, mode='min')
callbacks_list_1 = [checkpoint]

# training the model

start_time = time()
model_1.fit(X_train,y_train,batch_size= 1000, epochs= 250,
          validation_data=(X_valid,y_valid),
          #validation_split = 0.2,
         verbose =2,callbacks = callbacks_list_1)
end_time = time()

Train on 40392 samples, validate on 10098 samples
Epoch 1/250
 - 4s - loss: 4.4220 - acc: 0.6810 - val_loss: 2.4390 - val_acc: 0.7485

Epoch 00001: val_loss improved from inf to 2.43905, saving model to weights-improvement-best-model_1.hdf5
Epoch 2/250
 - 4s - loss: 1.6267 - acc: 0.7389 - val_loss: 1.0277 - val_acc: 0.7550

Epoch 00002: val_loss improved from 2.43905 to 1.02766, saving model to weights-improvement-best-model_1.hdf5
Epoch 3/250
 - 4s - loss: 0.9020 - acc: 0.7473 - val_loss: 0.8926 - val_acc: 0.7160

Epoch 00003: val_loss improved from 1.02766 to 0.89259, saving model to weights-improvement-best-model_1.hdf5
Epoch 4/250
 - 4s - loss: 0.8043 - acc: 0.7495 - val_loss: 0.7903 - val_acc: 0.7440

Epoch 00004: val_loss improved from 0.89259 to 0.79030, saving model to weights-improvement-best-model_1.hdf5
Epoch 5/250
 - 4s - loss: 0.7647 - acc: 0.7537 - val_loss: 0.8050 - val_acc: 0.7349

Epoch 00005: val_loss did not improve from 0.79030
Epoch 6/250
 - 4s - loss: 0.7497 - acc: 0.7524 - val_loss: 0.7242 - val_acc: 0.7619

Epoch 00006: val_loss improved from 0.79030 to 0.72423, saving model to weights-improvement-best-model_1.hdf5
Epoch 7/250
 - 4s - loss: 0.7340 - acc: 0.7573 - val_loss: 0.7066 - val_acc: 0.7680

Epoch 00007: val_loss improved from 0.72423 to 0.70662, saving model to weights-improvement-best-model_1.hdf5
Epoch 8/250
 - 4s - loss: 0.7204 - acc: 0.7596 - val_loss: 0.7035 - val_acc: 0.7660

Epoch 00008: val_loss improved from 0.70662 to 0.70354, saving model to weights-improvement-best-model_1.hdf5
Epoch 9/250
 - 4s - loss: 0.7163 - acc: 0.7592 - val_loss: 0.7149 - val_acc: 0.7532

Epoch 00009: val_loss did not improve from 0.70354
Epoch 10/250
 - 4s - loss: 0.7119 - acc: 0.7601 - val_loss: 0.7216 - val_acc: 0.7528

Epoch 00010: val_loss did not improve from 0.70354
Epoch 11/250
 - 4s - loss: 0.7058 - acc: 0.7603 - val_loss: 0.6915 - val_acc: 0.7682

Epoch 00011: val_loss improved from 0.70354 to 0.69147, saving model to weights-improvement-best-model_1.hdf5
Epoch 12/250
 - 4s - loss: 0.6997 - acc: 0.7640 - val_loss: 0.7199 - val_acc: 0.7530

Epoch 00012: val_loss did not improve from 0.69147
Epoch 13/250
 - 4s - loss: 0.6944 - acc: 0.7656 - val_loss: 0.7046 - val_acc: 0.7653

Epoch 00013: val_loss did not improve from 0.69147
Epoch 14/250
 - 4s - loss: 0.6904 - acc: 0.7655 - val_loss: 0.7193 - val_acc: 0.7454

Epoch 00014: val_loss did not improve from 0.69147
Epoch 15/250
 - 4s - loss: 0.6861 - acc: 0.7719 - val_loss: 0.6918 - val_acc: 0.7716

Epoch 00015: val_loss did not improve from 0.69147
Epoch 16/250
 - 4s - loss: 0.6835 - acc: 0.7733 - val_loss: 0.6876 - val_acc: 0.7649

Epoch 00016: val_loss improved from 0.69147 to 0.68756, saving model to weights-improvement-best-model_1.hdf5
Epoch 17/250
 - 4s - loss: 0.6787 - acc: 0.7754 - val_loss: 0.6760 - val_acc: 0.7778

Epoch 00017: val_loss improved from 0.68756 to 0.67596, saving model to weights-improvement-best-model_1.hdf5
Epoch 18/250
 - 4s - loss: 0.6775 - acc: 0.7763 - val_loss: 0.6823 - val_acc: 0.7703

Epoch 00018: val_loss did not improve from 0.67596
Epoch 19/250
 - 4s - loss: 0.6730 - acc: 0.7779 - val_loss: 0.6894 - val_acc: 0.7732

Epoch 00019: val_loss did not improve from 0.67596
Epoch 20/250
 - 4s - loss: 0.6701 - acc: 0.7798 - val_loss: 0.6727 - val_acc: 0.7801

Epoch 00020: val_loss improved from 0.67596 to 0.67267, saving model to weights-improvement-best-model_1.hdf5
Epoch 21/250
 - 4s - loss: 0.6681 - acc: 0.7823 - val_loss: 0.6764 - val_acc: 0.7726

Epoch 00021: val_loss did not improve from 0.67267
Epoch 22/250
 - 4s - loss: 0.6686 - acc: 0.7827 - val_loss: 0.6720 - val_acc: 0.7819

Epoch 00022: val_loss improved from 0.67267 to 0.67204, saving model to weights-improvement-best-model_1.hdf5
Epoch 23/250
 - 4s - loss: 0.6644 - acc: 0.7847 - val_loss: 0.6707 - val_acc: 0.7812

Epoch 00023: val_loss improved from 0.67204 to 0.67070, saving model to weights-improvement-best-model_1.hdf5
Epoch 24/250
 - 4s - loss: 0.6613 - acc: 0.7848 - val_loss: 0.6812 - val_acc: 0.7788

Epoch 00024: val_loss did not improve from 0.67070
Epoch 25/250
 - 4s - loss: 0.6625 - acc: 0.7838 - val_loss: 0.6917 - val_acc: 0.7753

Epoch 00025: val_loss did not improve from 0.67070
Epoch 26/250
 - 4s - loss: 0.6582 - acc: 0.7865 - val_loss: 0.6667 - val_acc: 0.7778

Epoch 00026: val_loss improved from 0.67070 to 0.66672, saving model to weights-improvement-best-model_1.hdf5
Epoch 27/250
 - 4s - loss: 0.6545 - acc: 0.7874 - val_loss: 0.6755 - val_acc: 0.7765

Epoch 00027: val_loss did not improve from 0.66672
Epoch 28/250
 - 4s - loss: 0.6552 - acc: 0.7889 - val_loss: 0.7000 - val_acc: 0.7702

Epoch 00028: val_loss did not improve from 0.66672
Epoch 29/250
 - 4s - loss: 0.6529 - acc: 0.7875 - val_loss: 0.6706 - val_acc: 0.7801

Epoch 00029: val_loss did not improve from 0.66672
Epoch 30/250
 - 4s - loss: 0.6499 - acc: 0.7905 - val_loss: 0.6612 - val_acc: 0.7839

Epoch 00030: val_loss improved from 0.66672 to 0.66121, saving model to weights-improvement-best-model_1.hdf5
Epoch 31/250
 - 4s - loss: 0.6514 - acc: 0.7881 - val_loss: 0.6723 - val_acc: 0.7837

Epoch 00031: val_loss did not improve from 0.66121
Epoch 32/250
 - 4s - loss: 0.6481 - acc: 0.7932 - val_loss: 0.6839 - val_acc: 0.7659

Epoch 00032: val_loss did not improve from 0.66121
Epoch 33/250
 - 4s - loss: 0.6477 - acc: 0.7914 - val_loss: 0.6653 - val_acc: 0.7826

Epoch 00033: val_loss did not improve from 0.66121
Epoch 34/250
 - 4s - loss: 0.6457 - acc: 0.7929 - val_loss: 0.6807 - val_acc: 0.7797

Epoch 00034: val_loss did not improve from 0.66121
Epoch 35/250
 - 4s - loss: 0.6465 - acc: 0.7913 - val_loss: 0.6593 - val_acc: 0.7867

Epoch 00035: val_loss improved from 0.66121 to 0.65928, saving model to weights-improvement-best-model_1.hdf5
Epoch 36/250
 - 4s - loss: 0.6422 - acc: 0.7934 - val_loss: 0.6988 - val_acc: 0.7727

Epoch 00036: val_loss did not improve from 0.65928
Epoch 37/250
 - 4s - loss: 0.6448 - acc: 0.7930 - val_loss: 0.6756 - val_acc: 0.7796

Epoch 00037: val_loss did not improve from 0.65928
Epoch 38/250
 - 4s - loss: 0.6415 - acc: 0.7942 - val_loss: 0.6672 - val_acc: 0.7787

Epoch 00038: val_loss did not improve from 0.65928
Epoch 39/250
 - 4s - loss: 0.6387 - acc: 0.7934 - val_loss: 0.6868 - val_acc: 0.7690

Epoch 00039: val_loss did not improve from 0.65928
Epoch 40/250
 - 4s - loss: 0.6400 - acc: 0.7945 - val_loss: 0.6667 - val_acc: 0.7789

Epoch 00040: val_loss did not improve from 0.65928
Epoch 41/250
 - 4s - loss: 0.6366 - acc: 0.7965 - val_loss: 0.6602 - val_acc: 0.7866

Epoch 00041: val_loss did not improve from 0.65928
Epoch 42/250
 - 4s - loss: 0.6354 - acc: 0.7981 - val_loss: 0.6775 - val_acc: 0.7835

Epoch 00042: val_loss did not improve from 0.65928
Epoch 43/250
 - 4s - loss: 0.6345 - acc: 0.7964 - val_loss: 0.6844 - val_acc: 0.7686

Epoch 00043: val_loss did not improve from 0.65928
Epoch 44/250
 - 4s - loss: 0.6355 - acc: 0.7964 - val_loss: 0.6804 - val_acc: 0.7701

Epoch 00044: val_loss did not improve from 0.65928
Epoch 45/250
 - 4s - loss: 0.6312 - acc: 0.7987 - val_loss: 0.6609 - val_acc: 0.7834

Epoch 00045: val_loss did not improve from 0.65928
Epoch 46/250
 - 5s - loss: 0.6325 - acc: 0.7971 - val_loss: 0.6741 - val_acc: 0.7858

Epoch 00046: val_loss did not improve from 0.65928
Epoch 47/250
 - 4s - loss: 0.6331 - acc: 0.7991 - val_loss: 0.6612 - val_acc: 0.7888

Epoch 00047: val_loss did not improve from 0.65928
Epoch 48/250
 - 4s - loss: 0.6331 - acc: 0.7990 - val_loss: 0.6633 - val_acc: 0.7856

Epoch 00048: val_loss did not improve from 0.65928
Epoch 49/250
 - 4s - loss: 0.6287 - acc: 0.7995 - val_loss: 0.6672 - val_acc: 0.7806

Epoch 00049: val_loss did not improve from 0.65928
Epoch 50/250
 - 4s - loss: 0.6242 - acc: 0.8027 - val_loss: 0.6813 - val_acc: 0.7710

Epoch 00050: val_loss did not improve from 0.65928
Epoch 51/250
 - 4s - loss: 0.6269 - acc: 0.8036 - val_loss: 0.7072 - val_acc: 0.7484

Epoch 00051: val_loss did not improve from 0.65928
Epoch 52/250
 - 4s - loss: 0.6251 - acc: 0.8044 - val_loss: 0.7071 - val_acc: 0.7747

Epoch 00052: val_loss did not improve from 0.65928
Epoch 53/250
 - 4s - loss: 0.6260 - acc: 0.8020 - val_loss: 0.6732 - val_acc: 0.7838

Epoch 00053: val_loss did not improve from 0.65928
Epoch 54/250
 - 4s - loss: 0.6245 - acc: 0.8024 - val_loss: 0.6703 - val_acc: 0.7859

Epoch 00054: val_loss did not improve from 0.65928
Epoch 55/250
 - 4s - loss: 0.6253 - acc: 0.8024 - val_loss: 0.6692 - val_acc: 0.7774

Epoch 00055: val_loss did not improve from 0.65928
Epoch 56/250
 - 4s - loss: 0.6245 - acc: 0.8032 - val_loss: 0.7009 - val_acc: 0.7569

Epoch 00056: val_loss did not improve from 0.65928
Epoch 57/250
 - 4s - loss: 0.6235 - acc: 0.8030 - val_loss: 0.6594 - val_acc: 0.7863

Epoch 00057: val_loss did not improve from 0.65928
Epoch 58/250
 - 4s - loss: 0.6213 - acc: 0.8037 - val_loss: 0.6613 - val_acc: 0.7837

Epoch 00058: val_loss did not improve from 0.65928
Epoch 59/250
 - 4s - loss: 0.6247 - acc: 0.8025 - val_loss: 0.6723 - val_acc: 0.7878

Epoch 00059: val_loss did not improve from 0.65928
Epoch 60/250
 - 4s - loss: 0.6228 - acc: 0.8022 - val_loss: 0.6675 - val_acc: 0.7903

Epoch 00060: val_loss did not improve from 0.65928
Epoch 61/250
 - 4s - loss: 0.6205 - acc: 0.8036 - val_loss: 0.6756 - val_acc: 0.7701

Epoch 00061: val_loss did not improve from 0.65928
Epoch 62/250
 - 4s - loss: 0.6152 - acc: 0.8075 - val_loss: 0.6689 - val_acc: 0.7768

Epoch 00062: val_loss did not improve from 0.65928
Epoch 63/250
 - 4s - loss: 0.6193 - acc: 0.8055 - val_loss: 0.6674 - val_acc: 0.7760

Epoch 00063: val_loss did not improve from 0.65928
Epoch 64/250
 - 4s - loss: 0.6187 - acc: 0.8050 - val_loss: 0.6621 - val_acc: 0.7806

Epoch 00064: val_loss did not improve from 0.65928
Epoch 65/250
 - 4s - loss: 0.6184 - acc: 0.8043 - val_loss: 0.6570 - val_acc: 0.7879

Epoch 00065: val_loss improved from 0.65928 to 0.65702, saving model to weights-improvement-best-model_1.hdf5
Epoch 66/250
 - 4s - loss: 0.6150 - acc: 0.8072 - val_loss: 0.6606 - val_acc: 0.7867

Epoch 00066: val_loss did not improve from 0.65702
Epoch 67/250
 - 4s - loss: 0.6117 - acc: 0.8092 - val_loss: 0.6728 - val_acc: 0.7798

Epoch 00067: val_loss did not improve from 0.65702
Epoch 68/250
 - 4s - loss: 0.6134 - acc: 0.8088 - val_loss: 0.7106 - val_acc: 0.7780

Epoch 00068: val_loss did not improve from 0.65702
Epoch 69/250
 - 4s - loss: 0.6176 - acc: 0.8064 - val_loss: 0.6668 - val_acc: 0.7881

Epoch 00069: val_loss did not improve from 0.65702
Epoch 70/250
 - 4s - loss: 0.6127 - acc: 0.8102 - val_loss: 0.6873 - val_acc: 0.7808

Epoch 00070: val_loss did not improve from 0.65702
Epoch 71/250
 - 4s - loss: 0.6143 - acc: 0.8091 - val_loss: 0.6600 - val_acc: 0.7868

Epoch 00071: val_loss did not improve from 0.65702
Epoch 72/250
 - 4s - loss: 0.6143 - acc: 0.8069 - val_loss: 0.6848 - val_acc: 0.7644

Epoch 00072: val_loss did not improve from 0.65702
Epoch 73/250
 - 4s - loss: 0.6069 - acc: 0.8100 - val_loss: 0.6659 - val_acc: 0.7788

Epoch 00073: val_loss did not improve from 0.65702
Epoch 74/250
 - 4s - loss: 0.6123 - acc: 0.8080 - val_loss: 0.6870 - val_acc: 0.7631

Epoch 00074: val_loss did not improve from 0.65702
Epoch 75/250
 - 4s - loss: 0.6146 - acc: 0.8067 - val_loss: 0.6937 - val_acc: 0.7606

Epoch 00075: val_loss did not improve from 0.65702
Epoch 76/250
 - 4s - loss: 0.6096 - acc: 0.8085 - val_loss: 0.6685 - val_acc: 0.7807

Epoch 00076: val_loss did not improve from 0.65702
Epoch 77/250
 - 4s - loss: 0.6134 - acc: 0.8092 - val_loss: 0.6628 - val_acc: 0.7859

Epoch 00077: val_loss did not improve from 0.65702
Epoch 78/250
 - 4s - loss: 0.6116 - acc: 0.8088 - val_loss: 0.6612 - val_acc: 0.7887

Epoch 00078: val_loss did not improve from 0.65702
Epoch 79/250
 - 4s - loss: 0.6083 - acc: 0.8108 - val_loss: 0.6785 - val_acc: 0.7890

Epoch 00079: val_loss did not improve from 0.65702
Epoch 80/250
 - 4s - loss: 0.6066 - acc: 0.8115 - val_loss: 0.6801 - val_acc: 0.7869

Epoch 00080: val_loss did not improve from 0.65702
Epoch 81/250
 - 4s - loss: 0.6082 - acc: 0.8118 - val_loss: 0.6742 - val_acc: 0.7871

Epoch 00081: val_loss did not improve from 0.65702
Epoch 82/250
 - 4s - loss: 0.6077 - acc: 0.8113 - val_loss: 0.7121 - val_acc: 0.7755

Epoch 00082: val_loss did not improve from 0.65702
Epoch 83/250
 - 4s - loss: 0.6078 - acc: 0.8098 - val_loss: 0.6638 - val_acc: 0.7824

Epoch 00083: val_loss did not improve from 0.65702
Epoch 84/250
 - 4s - loss: 0.6077 - acc: 0.8114 - val_loss: 0.6678 - val_acc: 0.7913

Epoch 00084: val_loss did not improve from 0.65702
Epoch 85/250
 - 4s - loss: 0.6039 - acc: 0.8128 - val_loss: 0.6741 - val_acc: 0.7757

Epoch 00085: val_loss did not improve from 0.65702
Epoch 86/250
 - 4s - loss: 0.6043 - acc: 0.8125 - val_loss: 0.6885 - val_acc: 0.7834

Epoch 00086: val_loss did not improve from 0.65702
Epoch 87/250
 - 4s - loss: 0.6029 - acc: 0.8137 - val_loss: 0.6719 - val_acc: 0.7888

Epoch 00087: val_loss did not improve from 0.65702
Epoch 88/250
 - 4s - loss: 0.6054 - acc: 0.8124 - val_loss: 0.6961 - val_acc: 0.7595

Epoch 00088: val_loss did not improve from 0.65702
Epoch 89/250
 - 4s - loss: 0.6026 - acc: 0.8126 - val_loss: 0.6799 - val_acc: 0.7734

Epoch 00089: val_loss did not improve from 0.65702
Epoch 90/250
 - 4s - loss: 0.5995 - acc: 0.8153 - val_loss: 0.7145 - val_acc: 0.7506

Epoch 00090: val_loss did not improve from 0.65702
Epoch 91/250
 - 4s - loss: 0.6036 - acc: 0.8140 - val_loss: 0.6644 - val_acc: 0.7840

Epoch 00091: val_loss did not improve from 0.65702
Epoch 92/250
 - 4s - loss: 0.6061 - acc: 0.8128 - val_loss: 0.6613 - val_acc: 0.7873

Epoch 00092: val_loss did not improve from 0.65702
Epoch 93/250
 - 4s - loss: 0.6035 - acc: 0.8128 - val_loss: 0.6805 - val_acc: 0.7735

Epoch 00093: val_loss did not improve from 0.65702
Epoch 94/250
 - 4s - loss: 0.6033 - acc: 0.8145 - val_loss: 0.6629 - val_acc: 0.7858

Epoch 00094: val_loss did not improve from 0.65702
Epoch 95/250
 - 4s - loss: 0.6045 - acc: 0.8135 - val_loss: 0.6646 - val_acc: 0.7889

Epoch 00095: val_loss did not improve from 0.65702
Epoch 96/250
 - 4s - loss: 0.6027 - acc: 0.8146 - val_loss: 0.6793 - val_acc: 0.7749

Epoch 00096: val_loss did not improve from 0.65702
Epoch 97/250
 - 4s - loss: 0.6016 - acc: 0.8135 - val_loss: 0.6845 - val_acc: 0.7748

Epoch 00097: val_loss did not improve from 0.65702
Epoch 98/250
 - 4s - loss: 0.6025 - acc: 0.8139 - val_loss: 0.6607 - val_acc: 0.7898

Epoch 00098: val_loss did not improve from 0.65702
Epoch 99/250
 - 4s - loss: 0.5978 - acc: 0.8139 - val_loss: 0.6992 - val_acc: 0.7609

Epoch 00099: val_loss did not improve from 0.65702
Epoch 100/250
 - 4s - loss: 0.6027 - acc: 0.8140 - val_loss: 0.6645 - val_acc: 0.7820

Epoch 00100: val_loss did not improve from 0.65702
Epoch 101/250
 - 4s - loss: 0.6003 - acc: 0.8146 - val_loss: 0.6751 - val_acc: 0.7882

Epoch 00101: val_loss did not improve from 0.65702
Epoch 102/250
 - 4s - loss: 0.5975 - acc: 0.8172 - val_loss: 0.6691 - val_acc: 0.7795

Epoch 00102: val_loss did not improve from 0.65702
Epoch 103/250
 - 4s - loss: 0.5995 - acc: 0.8151 - val_loss: 0.6974 - val_acc: 0.7836

Epoch 00103: val_loss did not improve from 0.65702
Epoch 104/250
 - 4s - loss: 0.5969 - acc: 0.8157 - val_loss: 0.6761 - val_acc: 0.7772

Epoch 00104: val_loss did not improve from 0.65702
Epoch 105/250
 - 4s - loss: 0.5993 - acc: 0.8155 - val_loss: 0.6748 - val_acc: 0.7788

Epoch 00105: val_loss did not improve from 0.65702
Epoch 106/250
 - 4s - loss: 0.5998 - acc: 0.8144 - val_loss: 0.6665 - val_acc: 0.7843

Epoch 00106: val_loss did not improve from 0.65702
Epoch 107/250
 - 4s - loss: 0.5960 - acc: 0.8163 - val_loss: 0.6602 - val_acc: 0.7848

Epoch 00107: val_loss did not improve from 0.65702
Epoch 108/250
 - 4s - loss: 0.5958 - acc: 0.8169 - val_loss: 0.6684 - val_acc: 0.7836

Epoch 00108: val_loss did not improve from 0.65702
Epoch 109/250
 - 4s - loss: 0.5945 - acc: 0.8181 - val_loss: 0.6733 - val_acc: 0.7776

Epoch 00109: val_loss did not improve from 0.65702
Epoch 110/250
 - 4s - loss: 0.5914 - acc: 0.8202 - val_loss: 0.6917 - val_acc: 0.7670

Epoch 00110: val_loss did not improve from 0.65702
Epoch 111/250
 - 4s - loss: 0.5980 - acc: 0.8162 - val_loss: 0.7066 - val_acc: 0.7833

Epoch 00111: val_loss did not improve from 0.65702
Epoch 112/250
 - 4s - loss: 0.5941 - acc: 0.8177 - val_loss: 0.6873 - val_acc: 0.7703

Epoch 00112: val_loss did not improve from 0.65702
Epoch 113/250
 - 4s - loss: 0.5965 - acc: 0.8173 - val_loss: 0.6670 - val_acc: 0.7821

Epoch 00113: val_loss did not improve from 0.65702
Epoch 114/250
 - 4s - loss: 0.5950 - acc: 0.8195 - val_loss: 0.6765 - val_acc: 0.7745

Epoch 00114: val_loss did not improve from 0.65702
Epoch 115/250
 - 4s - loss: 0.5932 - acc: 0.8196 - val_loss: 0.6653 - val_acc: 0.7895

Epoch 00115: val_loss did not improve from 0.65702
Epoch 116/250
 - 4s - loss: 0.5900 - acc: 0.8203 - val_loss: 0.6707 - val_acc: 0.7858

Epoch 00116: val_loss did not improve from 0.65702
Epoch 117/250
 - 4s - loss: 0.5950 - acc: 0.8187 - val_loss: 0.6738 - val_acc: 0.7860

Epoch 00117: val_loss did not improve from 0.65702
Epoch 118/250
 - 4s - loss: 0.5953 - acc: 0.8177 - val_loss: 0.6944 - val_acc: 0.7890

Epoch 00118: val_loss did not improve from 0.65702
Epoch 119/250
 - 4s - loss: 0.5938 - acc: 0.8175 - val_loss: 0.6872 - val_acc: 0.7896

Epoch 00119: val_loss did not improve from 0.65702
Epoch 120/250
 - 4s - loss: 0.5913 - acc: 0.8189 - val_loss: 0.7274 - val_acc: 0.7811

Epoch 00120: val_loss did not improve from 0.65702
Epoch 121/250
 - 4s - loss: 0.5927 - acc: 0.8188 - val_loss: 0.6707 - val_acc: 0.7810

Epoch 00121: val_loss did not improve from 0.65702
Epoch 122/250
 - 4s - loss: 0.5903 - acc: 0.8202 - val_loss: 0.6749 - val_acc: 0.7812

Epoch 00122: val_loss did not improve from 0.65702
Epoch 123/250
 - 4s - loss: 0.5950 - acc: 0.8214 - val_loss: 0.6780 - val_acc: 0.7703

Epoch 00123: val_loss did not improve from 0.65702
Epoch 124/250
 - 4s - loss: 0.5901 - acc: 0.8201 - val_loss: 0.6827 - val_acc: 0.7868

Epoch 00124: val_loss did not improve from 0.65702
Epoch 125/250
 - 4s - loss: 0.5896 - acc: 0.8211 - val_loss: 0.6751 - val_acc: 0.7869

Epoch 00125: val_loss did not improve from 0.65702
Epoch 126/250
 - 4s - loss: 0.5873 - acc: 0.8220 - val_loss: 0.7397 - val_acc: 0.7753

Epoch 00126: val_loss did not improve from 0.65702
Epoch 127/250
 - 4s - loss: 0.5906 - acc: 0.8211 - val_loss: 0.6698 - val_acc: 0.7855

Epoch 00127: val_loss did not improve from 0.65702
Epoch 128/250
 - 4s - loss: 0.5887 - acc: 0.8224 - val_loss: 0.6822 - val_acc: 0.7751

Epoch 00128: val_loss did not improve from 0.65702
Epoch 129/250
 - 4s - loss: 0.5874 - acc: 0.8213 - val_loss: 0.6745 - val_acc: 0.7893

Epoch 00129: val_loss did not improve from 0.65702
Epoch 130/250
 - 4s - loss: 0.5897 - acc: 0.8191 - val_loss: 0.6770 - val_acc: 0.7885

Epoch 00130: val_loss did not improve from 0.65702
Epoch 131/250
 - 4s - loss: 0.5881 - acc: 0.8209 - val_loss: 0.6698 - val_acc: 0.7802

Epoch 00131: val_loss did not improve from 0.65702
Epoch 132/250
 - 4s - loss: 0.5861 - acc: 0.8220 - val_loss: 0.6927 - val_acc: 0.7942

Epoch 00132: val_loss did not improve from 0.65702
Epoch 133/250
 - 4s - loss: 0.5914 - acc: 0.8198 - val_loss: 0.6803 - val_acc: 0.7822

Epoch 00133: val_loss did not improve from 0.65702
Epoch 134/250
 - 4s - loss: 0.5888 - acc: 0.8219 - val_loss: 0.6908 - val_acc: 0.7915

Epoch 00134: val_loss did not improve from 0.65702
Epoch 135/250
 - 4s - loss: 0.5850 - acc: 0.8227 - val_loss: 0.6747 - val_acc: 0.7838

Epoch 00135: val_loss did not improve from 0.65702
Epoch 136/250
 - 4s - loss: 0.5888 - acc: 0.8214 - val_loss: 0.7014 - val_acc: 0.7627

Epoch 00136: val_loss did not improve from 0.65702
Epoch 137/250
 - 5s - loss: 0.5872 - acc: 0.8221 - val_loss: 0.7058 - val_acc: 0.7602

Epoch 00137: val_loss did not improve from 0.65702
Epoch 138/250
 - 4s - loss: 0.5877 - acc: 0.8216 - val_loss: 0.6745 - val_acc: 0.7822

Epoch 00138: val_loss did not improve from 0.65702
Epoch 139/250
 - 4s - loss: 0.5889 - acc: 0.8218 - val_loss: 0.6836 - val_acc: 0.7788

Epoch 00139: val_loss did not improve from 0.65702
Epoch 140/250
 - 4s - loss: 0.5845 - acc: 0.8221 - val_loss: 0.6753 - val_acc: 0.7849

Epoch 00140: val_loss did not improve from 0.65702
Epoch 141/250
 - 4s - loss: 0.5908 - acc: 0.8200 - val_loss: 0.6945 - val_acc: 0.7745

Epoch 00141: val_loss did not improve from 0.65702
Epoch 142/250
 - 4s - loss: 0.5875 - acc: 0.8191 - val_loss: 0.6832 - val_acc: 0.7916

Epoch 00142: val_loss did not improve from 0.65702
Epoch 143/250
 - 4s - loss: 0.5873 - acc: 0.8225 - val_loss: 0.6813 - val_acc: 0.7774

Epoch 00143: val_loss did not improve from 0.65702
Epoch 144/250
 - 4s - loss: 0.5873 - acc: 0.8226 - val_loss: 0.7055 - val_acc: 0.7920

Epoch 00144: val_loss did not improve from 0.65702
Epoch 145/250
 - 4s - loss: 0.5838 - acc: 0.8237 - val_loss: 0.6789 - val_acc: 0.7844

Epoch 00145: val_loss did not improve from 0.65702
Epoch 146/250
 - 4s - loss: 0.5863 - acc: 0.8221 - val_loss: 0.6801 - val_acc: 0.7870

Epoch 00146: val_loss did not improve from 0.65702
Epoch 147/250
 - 4s - loss: 0.5825 - acc: 0.8233 - val_loss: 0.6975 - val_acc: 0.7888

Epoch 00147: val_loss did not improve from 0.65702
Epoch 148/250
 - 4s - loss: 0.5887 - acc: 0.8205 - val_loss: 0.6883 - val_acc: 0.7870

Epoch 00148: val_loss did not improve from 0.65702
Epoch 149/250
 - 4s - loss: 0.5863 - acc: 0.8195 - val_loss: 0.6727 - val_acc: 0.7911

Epoch 00149: val_loss did not improve from 0.65702
Epoch 150/250
 - 4s - loss: 0.5826 - acc: 0.8245 - val_loss: 0.6696 - val_acc: 0.7891

Epoch 00150: val_loss did not improve from 0.65702
Epoch 151/250
 - 4s - loss: 0.5873 - acc: 0.8208 - val_loss: 0.6960 - val_acc: 0.7914

Epoch 00151: val_loss did not improve from 0.65702
Epoch 152/250
 - 4s - loss: 0.5838 - acc: 0.8234 - val_loss: 0.6848 - val_acc: 0.7908

Epoch 00152: val_loss did not improve from 0.65702
Epoch 153/250
 - 4s - loss: 0.5850 - acc: 0.8233 - val_loss: 0.6950 - val_acc: 0.7708

Epoch 00153: val_loss did not improve from 0.65702
Epoch 154/250
 - 4s - loss: 0.5834 - acc: 0.8237 - val_loss: 0.6739 - val_acc: 0.7842

Epoch 00154: val_loss did not improve from 0.65702
Epoch 155/250
 - 4s - loss: 0.5803 - acc: 0.8250 - val_loss: 0.6959 - val_acc: 0.7923

Epoch 00155: val_loss did not improve from 0.65702
Epoch 156/250
 - 4s - loss: 0.5865 - acc: 0.8222 - val_loss: 0.6752 - val_acc: 0.7879

Epoch 00156: val_loss did not improve from 0.65702
Epoch 157/250
 - 4s - loss: 0.5781 - acc: 0.8257 - val_loss: 0.6930 - val_acc: 0.7645

Epoch 00157: val_loss did not improve from 0.65702
Epoch 158/250
 - 4s - loss: 0.5861 - acc: 0.8236 - val_loss: 0.6750 - val_acc: 0.7877

Epoch 00158: val_loss did not improve from 0.65702
Epoch 159/250
 - 4s - loss: 0.5836 - acc: 0.8253 - val_loss: 0.6828 - val_acc: 0.7782

Epoch 00159: val_loss did not improve from 0.65702
Epoch 160/250
 - 4s - loss: 0.5810 - acc: 0.8240 - val_loss: 0.6879 - val_acc: 0.7775

Epoch 00160: val_loss did not improve from 0.65702
Epoch 161/250
 - 4s - loss: 0.5819 - acc: 0.8233 - val_loss: 0.6908 - val_acc: 0.7671

Epoch 00161: val_loss did not improve from 0.65702
Epoch 162/250
 - 4s - loss: 0.5824 - acc: 0.8246 - val_loss: 0.6802 - val_acc: 0.7819

Epoch 00162: val_loss did not improve from 0.65702
Epoch 163/250
 - 5s - loss: 0.5823 - acc: 0.8238 - val_loss: 0.6772 - val_acc: 0.7833

Epoch 00163: val_loss did not improve from 0.65702
Epoch 164/250
 - 4s - loss: 0.5823 - acc: 0.8250 - val_loss: 0.6810 - val_acc: 0.7725

Epoch 00164: val_loss did not improve from 0.65702
Epoch 165/250
 - 5s - loss: 0.5820 - acc: 0.8240 - val_loss: 0.6692 - val_acc: 0.7858

Epoch 00165: val_loss did not improve from 0.65702
Epoch 166/250
 - 5s - loss: 0.5812 - acc: 0.8266 - val_loss: 0.6857 - val_acc: 0.7929

Epoch 00166: val_loss did not improve from 0.65702
Epoch 167/250
 - 4s - loss: 0.5819 - acc: 0.8240 - val_loss: 0.6911 - val_acc: 0.7915

Epoch 00167: val_loss did not improve from 0.65702
Epoch 168/250
 - 4s - loss: 0.5828 - acc: 0.8246 - val_loss: 0.6831 - val_acc: 0.7765

Epoch 00168: val_loss did not improve from 0.65702
Epoch 169/250
 - 5s - loss: 0.5812 - acc: 0.8243 - val_loss: 0.7069 - val_acc: 0.7653

Epoch 00169: val_loss did not improve from 0.65702
Epoch 170/250
 - 4s - loss: 0.5794 - acc: 0.8252 - val_loss: 0.6993 - val_acc: 0.7879

Epoch 00170: val_loss did not improve from 0.65702
Epoch 171/250
 - 4s - loss: 0.5835 - acc: 0.8246 - val_loss: 0.6932 - val_acc: 0.7883

Epoch 00171: val_loss did not improve from 0.65702
Epoch 172/250
 - 5s - loss: 0.5771 - acc: 0.8275 - val_loss: 0.7608 - val_acc: 0.7737

Epoch 00172: val_loss did not improve from 0.65702
Epoch 173/250
 - 4s - loss: 0.5787 - acc: 0.8258 - val_loss: 0.6883 - val_acc: 0.7930

Epoch 00173: val_loss did not improve from 0.65702
Epoch 174/250
 - 4s - loss: 0.5828 - acc: 0.8253 - val_loss: 0.6965 - val_acc: 0.7788

Epoch 00174: val_loss did not improve from 0.65702
Epoch 175/250
 - 4s - loss: 0.5814 - acc: 0.8258 - val_loss: 0.6972 - val_acc: 0.7892

Epoch 00175: val_loss did not improve from 0.65702
Epoch 176/250
 - 4s - loss: 0.5748 - acc: 0.8301 - val_loss: 0.6975 - val_acc: 0.7902

Epoch 00176: val_loss did not improve from 0.65702
Epoch 177/250
 - 4s - loss: 0.5802 - acc: 0.8256 - val_loss: 0.6756 - val_acc: 0.7825

Epoch 00177: val_loss did not improve from 0.65702
Epoch 178/250
 - 5s - loss: 0.5830 - acc: 0.8227 - val_loss: 0.6858 - val_acc: 0.7859

Epoch 00178: val_loss did not improve from 0.65702
Epoch 179/250
 - 6s - loss: 0.5803 - acc: 0.8250 - val_loss: 0.6781 - val_acc: 0.7780

Epoch 00179: val_loss did not improve from 0.65702
Epoch 180/250
 - 5s - loss: 0.5784 - acc: 0.8254 - val_loss: 0.7013 - val_acc: 0.7902

Epoch 00180: val_loss did not improve from 0.65702
Epoch 181/250
 - 4s - loss: 0.5793 - acc: 0.8249 - val_loss: 0.6755 - val_acc: 0.7853

Epoch 00181: val_loss did not improve from 0.65702
Epoch 182/250
 - 4s - loss: 0.5769 - acc: 0.8275 - val_loss: 0.7053 - val_acc: 0.7708

Epoch 00182: val_loss did not improve from 0.65702
Epoch 183/250
 - 4s - loss: 0.5798 - acc: 0.8249 - val_loss: 0.6885 - val_acc: 0.7744

Epoch 00183: val_loss did not improve from 0.65702
Epoch 184/250
 - 4s - loss: 0.5760 - acc: 0.8276 - val_loss: 0.6985 - val_acc: 0.7902

Epoch 00184: val_loss did not improve from 0.65702
Epoch 185/250
 - 4s - loss: 0.5770 - acc: 0.8280 - val_loss: 0.7102 - val_acc: 0.7883

Epoch 00185: val_loss did not improve from 0.65702
Epoch 186/250
 - 4s - loss: 0.5793 - acc: 0.8264 - val_loss: 0.6798 - val_acc: 0.7851

Epoch 00186: val_loss did not improve from 0.65702
Epoch 187/250
 - 4s - loss: 0.5753 - acc: 0.8277 - val_loss: 0.6803 - val_acc: 0.7893

Epoch 00187: val_loss did not improve from 0.65702
Epoch 188/250
 - 4s - loss: 0.5785 - acc: 0.8280 - val_loss: 0.7331 - val_acc: 0.7506

Epoch 00188: val_loss did not improve from 0.65702
Epoch 189/250
 - 4s - loss: 0.5778 - acc: 0.8258 - val_loss: 0.6888 - val_acc: 0.7878

Epoch 00189: val_loss did not improve from 0.65702
Epoch 190/250
 - 4s - loss: 0.5794 - acc: 0.8262 - val_loss: 0.7142 - val_acc: 0.7850

Epoch 00190: val_loss did not improve from 0.65702
Epoch 191/250
 - 4s - loss: 0.5735 - acc: 0.8288 - val_loss: 0.6927 - val_acc: 0.7895

Epoch 00191: val_loss did not improve from 0.65702
Epoch 192/250
 - 5s - loss: 0.5801 - acc: 0.8260 - val_loss: 0.6875 - val_acc: 0.7734

Epoch 00192: val_loss did not improve from 0.65702
Epoch 193/250
 - 4s - loss: 0.5736 - acc: 0.8285 - val_loss: 0.7012 - val_acc: 0.7653

Epoch 00193: val_loss did not improve from 0.65702
Epoch 194/250
 - 4s - loss: 0.5755 - acc: 0.8269 - val_loss: 0.6909 - val_acc: 0.7910

Epoch 00194: val_loss did not improve from 0.65702
Epoch 195/250
 - 5s - loss: 0.5786 - acc: 0.8258 - val_loss: 0.6909 - val_acc: 0.7893

Epoch 00195: val_loss did not improve from 0.65702
Epoch 196/250
 - 4s - loss: 0.5768 - acc: 0.8263 - val_loss: 0.7005 - val_acc: 0.7649

Epoch 00196: val_loss did not improve from 0.65702
Epoch 197/250
 - 4s - loss: 0.5737 - acc: 0.8293 - val_loss: 0.6828 - val_acc: 0.7759

Epoch 00197: val_loss did not improve from 0.65702
Epoch 198/250
 - 4s - loss: 0.5780 - acc: 0.8297 - val_loss: 0.6853 - val_acc: 0.7788

Epoch 00198: val_loss did not improve from 0.65702
Epoch 199/250
 - 4s - loss: 0.5778 - acc: 0.8261 - val_loss: 0.6752 - val_acc: 0.7900

Epoch 00199: val_loss did not improve from 0.65702
Epoch 200/250
 - 4s - loss: 0.5739 - acc: 0.8291 - val_loss: 0.7097 - val_acc: 0.7599

Epoch 00200: val_loss did not improve from 0.65702
Epoch 201/250
 - 4s - loss: 0.5777 - acc: 0.8275 - val_loss: 0.6968 - val_acc: 0.7854

Epoch 00201: val_loss did not improve from 0.65702
Epoch 202/250
 - 4s - loss: 0.5759 - acc: 0.8292 - val_loss: 0.6883 - val_acc: 0.7874

Epoch 00202: val_loss did not improve from 0.65702
Epoch 203/250
 - 4s - loss: 0.5748 - acc: 0.8285 - val_loss: 0.6921 - val_acc: 0.7873

Epoch 00203: val_loss did not improve from 0.65702
Epoch 204/250
 - 4s - loss: 0.5786 - acc: 0.8255 - val_loss: 0.6754 - val_acc: 0.7851

Epoch 00204: val_loss did not improve from 0.65702
Epoch 205/250
 - 4s - loss: 0.5681 - acc: 0.8309 - val_loss: 0.7676 - val_acc: 0.7805

Epoch 00205: val_loss did not improve from 0.65702
Epoch 206/250
 - 4s - loss: 0.5778 - acc: 0.8280 - val_loss: 0.6804 - val_acc: 0.7891

Epoch 00206: val_loss did not improve from 0.65702
Epoch 207/250
 - 4s - loss: 0.5744 - acc: 0.8278 - val_loss: 0.7137 - val_acc: 0.7888

Epoch 00207: val_loss did not improve from 0.65702
Epoch 208/250
 - 4s - loss: 0.5760 - acc: 0.8275 - val_loss: 0.7175 - val_acc: 0.7557

Epoch 00208: val_loss did not improve from 0.65702
Epoch 209/250
 - 4s - loss: 0.5718 - acc: 0.8305 - val_loss: 0.6826 - val_acc: 0.7873

Epoch 00209: val_loss did not improve from 0.65702
Epoch 210/250
 - 4s - loss: 0.5721 - acc: 0.8290 - val_loss: 0.6976 - val_acc: 0.7804

Epoch 00210: val_loss did not improve from 0.65702
Epoch 211/250
 - 4s - loss: 0.5722 - acc: 0.8311 - val_loss: 0.6872 - val_acc: 0.7893

Epoch 00211: val_loss did not improve from 0.65702
Epoch 212/250
 - 4s - loss: 0.5748 - acc: 0.8274 - val_loss: 0.6891 - val_acc: 0.7828

Epoch 00212: val_loss did not improve from 0.65702
Epoch 213/250
 - 4s - loss: 0.5730 - acc: 0.8300 - val_loss: 0.6824 - val_acc: 0.7875

Epoch 00213: val_loss did not improve from 0.65702
Epoch 214/250
 - 5s - loss: 0.5741 - acc: 0.8289 - val_loss: 0.6836 - val_acc: 0.7787

Epoch 00214: val_loss did not improve from 0.65702
Epoch 215/250
 - 4s - loss: 0.5741 - acc: 0.8309 - val_loss: 0.6972 - val_acc: 0.7898

Epoch 00215: val_loss did not improve from 0.65702
Epoch 216/250
 - 5s - loss: 0.5768 - acc: 0.8292 - val_loss: 0.6932 - val_acc: 0.7811

Epoch 00216: val_loss did not improve from 0.65702
Epoch 217/250
 - 5s - loss: 0.5718 - acc: 0.8300 - val_loss: 0.6903 - val_acc: 0.7752

Epoch 00217: val_loss did not improve from 0.65702
Epoch 218/250
 - 4s - loss: 0.5729 - acc: 0.8283 - val_loss: 0.7024 - val_acc: 0.7897

Epoch 00218: val_loss did not improve from 0.65702
Epoch 219/250
 - 4s - loss: 0.5697 - acc: 0.8300 - val_loss: 0.6943 - val_acc: 0.7801

Epoch 00219: val_loss did not improve from 0.65702
Epoch 220/250
 - 5s - loss: 0.5723 - acc: 0.8306 - val_loss: 0.6905 - val_acc: 0.7730

Epoch 00220: val_loss did not improve from 0.65702
Epoch 221/250
 - 5s - loss: 0.5732 - acc: 0.8292 - val_loss: 0.6935 - val_acc: 0.7797

Epoch 00221: val_loss did not improve from 0.65702
Epoch 222/250
 - 5s - loss: 0.5721 - acc: 0.8297 - val_loss: 0.7087 - val_acc: 0.7895

Epoch 00222: val_loss did not improve from 0.65702
Epoch 223/250
 - 4s - loss: 0.5701 - acc: 0.8315 - val_loss: 0.7228 - val_acc: 0.7876

Epoch 00223: val_loss did not improve from 0.65702
Epoch 224/250
 - 4s - loss: 0.5766 - acc: 0.8271 - val_loss: 0.7164 - val_acc: 0.7896

Epoch 00224: val_loss did not improve from 0.65702
Epoch 225/250
 - 4s - loss: 0.5740 - acc: 0.8291 - val_loss: 0.6929 - val_acc: 0.7878

Epoch 00225: val_loss did not improve from 0.65702
Epoch 226/250
 - 5s - loss: 0.5734 - acc: 0.8284 - val_loss: 0.6830 - val_acc: 0.7833

Epoch 00226: val_loss did not improve from 0.65702
Epoch 227/250
 - 4s - loss: 0.5727 - acc: 0.8289 - val_loss: 0.6832 - val_acc: 0.7852

Epoch 00227: val_loss did not improve from 0.65702
Epoch 228/250
 - 4s - loss: 0.5740 - acc: 0.8297 - val_loss: 0.6780 - val_acc: 0.7896

Epoch 00228: val_loss did not improve from 0.65702
Epoch 229/250
 - 4s - loss: 0.5696 - acc: 0.8318 - val_loss: 0.7061 - val_acc: 0.7809

Epoch 00229: val_loss did not improve from 0.65702
Epoch 230/250
 - 4s - loss: 0.5740 - acc: 0.8315 - val_loss: 0.6906 - val_acc: 0.7934

Epoch 00230: val_loss did not improve from 0.65702
Epoch 231/250
 - 5s - loss: 0.5714 - acc: 0.8308 - val_loss: 0.6869 - val_acc: 0.7806

Epoch 00231: val_loss did not improve from 0.65702
Epoch 232/250
 - 4s - loss: 0.5681 - acc: 0.8331 - val_loss: 0.6855 - val_acc: 0.7892

Epoch 00232: val_loss did not improve from 0.65702
Epoch 233/250
 - 4s - loss: 0.5686 - acc: 0.8318 - val_loss: 0.6915 - val_acc: 0.7920

Epoch 00233: val_loss did not improve from 0.65702
Epoch 234/250
 - 4s - loss: 0.5719 - acc: 0.8319 - val_loss: 0.7078 - val_acc: 0.7708

Epoch 00234: val_loss did not improve from 0.65702
Epoch 235/250
 - 4s - loss: 0.5718 - acc: 0.8309 - val_loss: 0.7037 - val_acc: 0.7921

Epoch 00235: val_loss did not improve from 0.65702
Epoch 236/250
 - 5s - loss: 0.5698 - acc: 0.8312 - val_loss: 0.7048 - val_acc: 0.7644

Epoch 00236: val_loss did not improve from 0.65702
Epoch 237/250
 - 5s - loss: 0.5734 - acc: 0.8303 - val_loss: 0.6826 - val_acc: 0.7813

Epoch 00237: val_loss did not improve from 0.65702
Epoch 238/250
 - 5s - loss: 0.5682 - acc: 0.8320 - val_loss: 0.7631 - val_acc: 0.7467

Epoch 00238: val_loss did not improve from 0.65702
Epoch 239/250
 - 5s - loss: 0.5752 - acc: 0.8287 - val_loss: 0.6881 - val_acc: 0.7903

Epoch 00239: val_loss did not improve from 0.65702
Epoch 240/250
 - 5s - loss: 0.5690 - acc: 0.8314 - val_loss: 0.7031 - val_acc: 0.7908

Epoch 00240: val_loss did not improve from 0.65702
Epoch 241/250
 - 4s - loss: 0.5728 - acc: 0.8286 - val_loss: 0.6775 - val_acc: 0.7859

Epoch 00241: val_loss did not improve from 0.65702
Epoch 242/250
 - 5s - loss: 0.5687 - acc: 0.8318 - val_loss: 0.6821 - val_acc: 0.7900

Epoch 00242: val_loss did not improve from 0.65702
Epoch 243/250
 - 5s - loss: 0.5702 - acc: 0.8296 - val_loss: 0.6919 - val_acc: 0.7932

Epoch 00243: val_loss did not improve from 0.65702
Epoch 244/250
 - 4s - loss: 0.5710 - acc: 0.8305 - val_loss: 0.6828 - val_acc: 0.7817

Epoch 00244: val_loss did not improve from 0.65702
Epoch 245/250
 - 5s - loss: 0.5696 - acc: 0.8310 - val_loss: 0.6866 - val_acc: 0.7808

Epoch 00245: val_loss did not improve from 0.65702
Epoch 246/250
 - 4s - loss: 0.5707 - acc: 0.8297 - val_loss: 0.6880 - val_acc: 0.7865

Epoch 00246: val_loss did not improve from 0.65702
Epoch 247/250
 - 5s - loss: 0.5717 - acc: 0.8296 - val_loss: 0.7251 - val_acc: 0.7577

Epoch 00247: val_loss did not improve from 0.65702
Epoch 248/250
 - 5s - loss: 0.5715 - acc: 0.8299 - val_loss: 0.6897 - val_acc: 0.7847

Epoch 00248: val_loss did not improve from 0.65702
Epoch 249/250
 - 5s - loss: 0.5684 - acc: 0.8316 - val_loss: 0.7144 - val_acc: 0.7870

Epoch 00249: val_loss did not improve from 0.65702
Epoch 250/250
 - 4s - loss: 0.5693 - acc: 0.8295 - val_loss: 0.6912 - val_acc: 0.7876

Epoch 00250: val_loss did not improve from 0.65702

# trying another architechture : model_2 with different optimizer
model_2 = Sequential()
model_2.add(Dense(1024, kernel_regularizer=regularizers.l2(.001),input_dim=X_train.shape[1]))
model_2.add(Activation('relu'))
model_2.add(Dropout(0.2))
model_2.add(Dense(512,kernel_regularizer=regularizers.l2(.001)))
model_2.add(Activation('relu'))
model_2.add(Dropout(0.2))
model_2.add(Dense(128,kernel_regularizer=regularizers.l1_l2(.001)))
model_2.add(Activation('relu'))
model_2.add(Dropout(0.3))
model_2.add(Dense(32, kernel_regularizer=regularizers.l2(.001)))
model_2.add(Activation('relu'))
model_2.add(Dense(3, activation = 'softmax'))

model_2.summary()

_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
dense_53 (Dense)             (None, 1024)              140288    
_________________________________________________________________
activation_42 (Activation)   (None, 1024)              0         
_________________________________________________________________
dropout_32 (Dropout)         (None, 1024)              0         
_________________________________________________________________
dense_54 (Dense)             (None, 512)               524800    
_________________________________________________________________
activation_43 (Activation)   (None, 512)               0         
_________________________________________________________________
dropout_33 (Dropout)         (None, 512)               0         
_________________________________________________________________
dense_55 (Dense)             (None, 128)               65664     
_________________________________________________________________
activation_44 (Activation)   (None, 128)               0         
_________________________________________________________________
dropout_34 (Dropout)         (None, 128)               0         
_________________________________________________________________
dense_56 (Dense)             (None, 32)                4128      
_________________________________________________________________
activation_45 (Activation)   (None, 32)                0         
_________________________________________________________________
dense_57 (Dense)             (None, 3)                 99        
=================================================================
Total params: 734,979
Trainable params: 734,979
Non-trainable params: 0
_________________________________________________________________

# compile the model_2
model_2.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])

# checkpoint for model_2
filepath="weights-improvement-best-model_2.hdf5"
checkpoint = ModelCheckpoint(filepath, monitor='val_loss', verbose=1, save_best_only=True, mode='min')
callbacks_list_1 = [checkpoint]

# training the model

start_time_2 = time()
model_2.fit(X_train,y_train,batch_size= 1000, epochs= 100,
          validation_data=(X_valid,y_valid),
          #validation_split = 0.2,
         verbose =2,callbacks = callbacks_list_1)
end_time_2 = time()

Train on 40392 samples, validate on 10098 samples
Epoch 1/100
 - 4s - loss: 4.7386 - acc: 0.6953 - val_loss: 2.7434 - val_acc: 0.7546

Epoch 00001: val_loss improved from inf to 2.74337, saving model to weights-improvement-best-model_2.hdf5
Epoch 2/100
 - 3s - loss: 1.8369 - acc: 0.7472 - val_loss: 1.1403 - val_acc: 0.7604

Epoch 00002: val_loss improved from 2.74337 to 1.14033, saving model to weights-improvement-best-model_2.hdf5
Epoch 3/100
 - 3s - loss: 0.9244 - acc: 0.7516 - val_loss: 0.7856 - val_acc: 0.7627

Epoch 00003: val_loss improved from 1.14033 to 0.78558, saving model to weights-improvement-best-model_2.hdf5
Epoch 4/100
 - 3s - loss: 0.7643 - acc: 0.7545 - val_loss: 0.7230 - val_acc: 0.7645

Epoch 00004: val_loss improved from 0.78558 to 0.72303, saving model to weights-improvement-best-model_2.hdf5
Epoch 5/100
 - 3s - loss: 0.7249 - acc: 0.7569 - val_loss: 0.7024 - val_acc: 0.7645

Epoch 00005: val_loss improved from 0.72303 to 0.70244, saving model to weights-improvement-best-model_2.hdf5
Epoch 6/100
 - 3s - loss: 0.7095 - acc: 0.7586 - val_loss: 0.6890 - val_acc: 0.7671

Epoch 00006: val_loss improved from 0.70244 to 0.68902, saving model to weights-improvement-best-model_2.hdf5
Epoch 7/100
 - 3s - loss: 0.6974 - acc: 0.7600 - val_loss: 0.6795 - val_acc: 0.7683

Epoch 00007: val_loss improved from 0.68902 to 0.67950, saving model to weights-improvement-best-model_2.hdf5
Epoch 8/100
 - 3s - loss: 0.6861 - acc: 0.7617 - val_loss: 0.6728 - val_acc: 0.7645

Epoch 00008: val_loss improved from 0.67950 to 0.67283, saving model to weights-improvement-best-model_2.hdf5
Epoch 9/100
 - 3s - loss: 0.6775 - acc: 0.7648 - val_loss: 0.6667 - val_acc: 0.7704

Epoch 00009: val_loss improved from 0.67283 to 0.66671, saving model to weights-improvement-best-model_2.hdf5
Epoch 10/100
 - 3s - loss: 0.6724 - acc: 0.7664 - val_loss: 0.6622 - val_acc: 0.7707

Epoch 00010: val_loss improved from 0.66671 to 0.66225, saving model to weights-improvement-best-model_2.hdf5
Epoch 11/100
 - 3s - loss: 0.6685 - acc: 0.7672 - val_loss: 0.6586 - val_acc: 0.7728

Epoch 00011: val_loss improved from 0.66225 to 0.65859, saving model to weights-improvement-best-model_2.hdf5
Epoch 12/100
 - 3s - loss: 0.6631 - acc: 0.7707 - val_loss: 0.6571 - val_acc: 0.7716

Epoch 00012: val_loss improved from 0.65859 to 0.65707, saving model to weights-improvement-best-model_2.hdf5
Epoch 13/100
 - 3s - loss: 0.6584 - acc: 0.7716 - val_loss: 0.6565 - val_acc: 0.7725

Epoch 00013: val_loss improved from 0.65707 to 0.65646, saving model to weights-improvement-best-model_2.hdf5
Epoch 14/100
 - 3s - loss: 0.6549 - acc: 0.7722 - val_loss: 0.6553 - val_acc: 0.7685

Epoch 00014: val_loss improved from 0.65646 to 0.65529, saving model to weights-improvement-best-model_2.hdf5
Epoch 15/100
 - 3s - loss: 0.6520 - acc: 0.7733 - val_loss: 0.6522 - val_acc: 0.7686

Epoch 00015: val_loss improved from 0.65529 to 0.65219, saving model to weights-improvement-best-model_2.hdf5
Epoch 16/100
 - 3s - loss: 0.6499 - acc: 0.7731 - val_loss: 0.6491 - val_acc: 0.7726

Epoch 00016: val_loss improved from 0.65219 to 0.64907, saving model to weights-improvement-best-model_2.hdf5
Epoch 17/100
 - 3s - loss: 0.6462 - acc: 0.7757 - val_loss: 0.6537 - val_acc: 0.7731

Epoch 00017: val_loss did not improve from 0.64907
Epoch 18/100
 - 3s - loss: 0.6452 - acc: 0.7762 - val_loss: 0.6455 - val_acc: 0.7713

Epoch 00018: val_loss improved from 0.64907 to 0.64546, saving model to weights-improvement-best-model_2.hdf5
Epoch 19/100
 - 3s - loss: 0.6410 - acc: 0.7803 - val_loss: 0.6472 - val_acc: 0.7748

Epoch 00019: val_loss did not improve from 0.64546
Epoch 20/100
 - 3s - loss: 0.6383 - acc: 0.7802 - val_loss: 0.6473 - val_acc: 0.7766

Epoch 00020: val_loss did not improve from 0.64546
Epoch 21/100
 - 3s - loss: 0.6362 - acc: 0.7798 - val_loss: 0.6400 - val_acc: 0.7765

Epoch 00021: val_loss improved from 0.64546 to 0.64004, saving model to weights-improvement-best-model_2.hdf5
Epoch 22/100
 - 3s - loss: 0.6354 - acc: 0.7825 - val_loss: 0.6409 - val_acc: 0.7752

Epoch 00022: val_loss did not improve from 0.64004
Epoch 23/100
 - 3s - loss: 0.6316 - acc: 0.7840 - val_loss: 0.6389 - val_acc: 0.7801

Epoch 00023: val_loss improved from 0.64004 to 0.63894, saving model to weights-improvement-best-model_2.hdf5
Epoch 24/100
 - 3s - loss: 0.6323 - acc: 0.7849 - val_loss: 0.6399 - val_acc: 0.7784

Epoch 00024: val_loss did not improve from 0.63894
Epoch 25/100
 - 3s - loss: 0.6308 - acc: 0.7873 - val_loss: 0.6446 - val_acc: 0.7771

Epoch 00025: val_loss did not improve from 0.63894
Epoch 26/100
 - 3s - loss: 0.6255 - acc: 0.7861 - val_loss: 0.6396 - val_acc: 0.7742

Epoch 00026: val_loss did not improve from 0.63894
Epoch 27/100
 - 3s - loss: 0.6256 - acc: 0.7867 - val_loss: 0.6373 - val_acc: 0.7819

Epoch 00027: val_loss improved from 0.63894 to 0.63731, saving model to weights-improvement-best-model_2.hdf5
Epoch 28/100
 - 3s - loss: 0.6246 - acc: 0.7903 - val_loss: 0.6418 - val_acc: 0.7790

Epoch 00028: val_loss did not improve from 0.63731
Epoch 29/100
 - 3s - loss: 0.6263 - acc: 0.7887 - val_loss: 0.6422 - val_acc: 0.7831

Epoch 00029: val_loss did not improve from 0.63731
Epoch 30/100
 - 3s - loss: 0.6214 - acc: 0.7906 - val_loss: 0.6363 - val_acc: 0.7822

Epoch 00030: val_loss improved from 0.63731 to 0.63635, saving model to weights-improvement-best-model_2.hdf5
Epoch 31/100
 - 3s - loss: 0.6184 - acc: 0.7921 - val_loss: 0.6373 - val_acc: 0.7823

Epoch 00031: val_loss did not improve from 0.63635
Epoch 32/100
 - 3s - loss: 0.6166 - acc: 0.7956 - val_loss: 0.6399 - val_acc: 0.7785

Epoch 00032: val_loss did not improve from 0.63635
Epoch 33/100
 - 3s - loss: 0.6156 - acc: 0.7917 - val_loss: 0.6389 - val_acc: 0.7807

Epoch 00033: val_loss did not improve from 0.63635
Epoch 34/100
 - 3s - loss: 0.6156 - acc: 0.7937 - val_loss: 0.6335 - val_acc: 0.7830

Epoch 00034: val_loss improved from 0.63635 to 0.63350, saving model to weights-improvement-best-model_2.hdf5
Epoch 35/100
 - 3s - loss: 0.6130 - acc: 0.7959 - val_loss: 0.6370 - val_acc: 0.7839

Epoch 00035: val_loss did not improve from 0.63350
Epoch 36/100
 - 3s - loss: 0.6160 - acc: 0.7951 - val_loss: 0.6337 - val_acc: 0.7807

Epoch 00036: val_loss did not improve from 0.63350
Epoch 37/100
 - 3s - loss: 0.6129 - acc: 0.7945 - val_loss: 0.6350 - val_acc: 0.7821

Epoch 00037: val_loss did not improve from 0.63350
Epoch 38/100
 - 3s - loss: 0.6077 - acc: 0.7980 - val_loss: 0.6348 - val_acc: 0.7832

Epoch 00038: val_loss did not improve from 0.63350
Epoch 39/100
 - 3s - loss: 0.6131 - acc: 0.7951 - val_loss: 0.6365 - val_acc: 0.7840

Epoch 00039: val_loss did not improve from 0.63350
Epoch 40/100
 - 3s - loss: 0.6096 - acc: 0.7966 - val_loss: 0.6311 - val_acc: 0.7836

Epoch 00040: val_loss improved from 0.63350 to 0.63108, saving model to weights-improvement-best-model_2.hdf5
Epoch 41/100
 - 3s - loss: 0.6060 - acc: 0.7990 - val_loss: 0.6357 - val_acc: 0.7834

Epoch 00041: val_loss did not improve from 0.63108
Epoch 42/100
 - 3s - loss: 0.6083 - acc: 0.7969 - val_loss: 0.6404 - val_acc: 0.7799

Epoch 00042: val_loss did not improve from 0.63108
Epoch 43/100
 - 3s - loss: 0.6079 - acc: 0.7987 - val_loss: 0.6319 - val_acc: 0.7867

Epoch 00043: val_loss did not improve from 0.63108
Epoch 44/100
 - 3s - loss: 0.6011 - acc: 0.8004 - val_loss: 0.6356 - val_acc: 0.7834

Epoch 00044: val_loss did not improve from 0.63108
Epoch 45/100
 - 3s - loss: 0.6025 - acc: 0.8000 - val_loss: 0.6314 - val_acc: 0.7878

Epoch 00045: val_loss did not improve from 0.63108
Epoch 46/100
 - 3s - loss: 0.6037 - acc: 0.8005 - val_loss: 0.6275 - val_acc: 0.7870

Epoch 00046: val_loss improved from 0.63108 to 0.62755, saving model to weights-improvement-best-model_2.hdf5
Epoch 47/100
 - 3s - loss: 0.6007 - acc: 0.8017 - val_loss: 0.6392 - val_acc: 0.7806

Epoch 00047: val_loss did not improve from 0.62755
Epoch 48/100
 - 3s - loss: 0.6028 - acc: 0.7991 - val_loss: 0.6389 - val_acc: 0.7831

Epoch 00048: val_loss did not improve from 0.62755
Epoch 49/100
 - 3s - loss: 0.5993 - acc: 0.8029 - val_loss: 0.6344 - val_acc: 0.7859

Epoch 00049: val_loss did not improve from 0.62755
Epoch 50/100
 - 3s - loss: 0.6014 - acc: 0.8007 - val_loss: 0.6295 - val_acc: 0.7858

Epoch 00050: val_loss did not improve from 0.62755
Epoch 51/100
 - 3s - loss: 0.5992 - acc: 0.8036 - val_loss: 0.6347 - val_acc: 0.7830

Epoch 00051: val_loss did not improve from 0.62755
Epoch 52/100
 - 3s - loss: 0.5967 - acc: 0.8039 - val_loss: 0.6336 - val_acc: 0.7816

Epoch 00052: val_loss did not improve from 0.62755
Epoch 53/100
 - 3s - loss: 0.5972 - acc: 0.8033 - val_loss: 0.6362 - val_acc: 0.7859

Epoch 00053: val_loss did not improve from 0.62755
Epoch 54/100
 - 3s - loss: 0.5971 - acc: 0.8023 - val_loss: 0.6368 - val_acc: 0.7841

Epoch 00054: val_loss did not improve from 0.62755
Epoch 55/100
 - 3s - loss: 0.5984 - acc: 0.8018 - val_loss: 0.6307 - val_acc: 0.7855

Epoch 00055: val_loss did not improve from 0.62755
Epoch 56/100
 - 3s - loss: 0.5943 - acc: 0.8044 - val_loss: 0.6306 - val_acc: 0.7873

Epoch 00056: val_loss did not improve from 0.62755
Epoch 57/100
 - 3s - loss: 0.5956 - acc: 0.8055 - val_loss: 0.6339 - val_acc: 0.7862

Epoch 00057: val_loss did not improve from 0.62755
Epoch 58/100
 - 3s - loss: 0.5934 - acc: 0.8048 - val_loss: 0.6357 - val_acc: 0.7795

Epoch 00058: val_loss did not improve from 0.62755
Epoch 59/100
 - 3s - loss: 0.5899 - acc: 0.8086 - val_loss: 0.6326 - val_acc: 0.7893

Epoch 00059: val_loss did not improve from 0.62755
Epoch 60/100
 - 3s - loss: 0.5914 - acc: 0.8080 - val_loss: 0.6315 - val_acc: 0.7868

Epoch 00060: val_loss did not improve from 0.62755
Epoch 61/100
 - 3s - loss: 0.5913 - acc: 0.8080 - val_loss: 0.6351 - val_acc: 0.7857

Epoch 00061: val_loss did not improve from 0.62755
Epoch 62/100
 - 3s - loss: 0.5920 - acc: 0.8075 - val_loss: 0.6375 - val_acc: 0.7832

Epoch 00062: val_loss did not improve from 0.62755
Epoch 63/100
 - 3s - loss: 0.5910 - acc: 0.8077 - val_loss: 0.6369 - val_acc: 0.7868

Epoch 00063: val_loss did not improve from 0.62755
Epoch 64/100
 - 3s - loss: 0.5902 - acc: 0.8081 - val_loss: 0.6314 - val_acc: 0.7870

Epoch 00064: val_loss did not improve from 0.62755
Epoch 65/100
 - 3s - loss: 0.5871 - acc: 0.8094 - val_loss: 0.6318 - val_acc: 0.7850

Epoch 00065: val_loss did not improve from 0.62755
Epoch 66/100
 - 3s - loss: 0.5860 - acc: 0.8113 - val_loss: 0.6305 - val_acc: 0.7903

Epoch 00066: val_loss did not improve from 0.62755
Epoch 67/100
 - 3s - loss: 0.5869 - acc: 0.8099 - val_loss: 0.6307 - val_acc: 0.7890

Epoch 00067: val_loss did not improve from 0.62755
Epoch 68/100
 - 3s - loss: 0.5845 - acc: 0.8109 - val_loss: 0.6411 - val_acc: 0.7841

Epoch 00068: val_loss did not improve from 0.62755
Epoch 69/100
 - 3s - loss: 0.5864 - acc: 0.8092 - val_loss: 0.6371 - val_acc: 0.7898

Epoch 00069: val_loss did not improve from 0.62755
Epoch 70/100
 - 3s - loss: 0.5850 - acc: 0.8116 - val_loss: 0.6301 - val_acc: 0.7883

Epoch 00070: val_loss did not improve from 0.62755
Epoch 71/100
 - 3s - loss: 0.5822 - acc: 0.8122 - val_loss: 0.6317 - val_acc: 0.7876

Epoch 00071: val_loss did not improve from 0.62755
Epoch 72/100
 - 3s - loss: 0.5818 - acc: 0.8133 - val_loss: 0.6328 - val_acc: 0.7895

Epoch 00072: val_loss did not improve from 0.62755
Epoch 73/100
 - 3s - loss: 0.5810 - acc: 0.8133 - val_loss: 0.6326 - val_acc: 0.7862

Epoch 00073: val_loss did not improve from 0.62755
Epoch 74/100
 - 3s - loss: 0.5802 - acc: 0.8144 - val_loss: 0.6296 - val_acc: 0.7876

Epoch 00074: val_loss did not improve from 0.62755
Epoch 75/100
 - 3s - loss: 0.5811 - acc: 0.8128 - val_loss: 0.6306 - val_acc: 0.7909

Epoch 00075: val_loss did not improve from 0.62755
Epoch 76/100
 - 3s - loss: 0.5774 - acc: 0.8166 - val_loss: 0.6420 - val_acc: 0.7786

Epoch 00076: val_loss did not improve from 0.62755
Epoch 77/100
 - 3s - loss: 0.5811 - acc: 0.8131 - val_loss: 0.6331 - val_acc: 0.7875

Epoch 00077: val_loss did not improve from 0.62755
Epoch 78/100
 - 3s - loss: 0.5788 - acc: 0.8144 - val_loss: 0.6375 - val_acc: 0.7864

Epoch 00078: val_loss did not improve from 0.62755
Epoch 79/100
 - 3s - loss: 0.5773 - acc: 0.8154 - val_loss: 0.6368 - val_acc: 0.7895

Epoch 00079: val_loss did not improve from 0.62755
Epoch 80/100
 - 3s - loss: 0.5802 - acc: 0.8133 - val_loss: 0.6348 - val_acc: 0.7826

Epoch 00080: val_loss did not improve from 0.62755
Epoch 81/100
 - 3s - loss: 0.5771 - acc: 0.8154 - val_loss: 0.6398 - val_acc: 0.7827

Epoch 00081: val_loss did not improve from 0.62755
Epoch 82/100
 - 3s - loss: 0.5803 - acc: 0.8131 - val_loss: 0.6358 - val_acc: 0.7855

Epoch 00082: val_loss did not improve from 0.62755
Epoch 83/100
 - 3s - loss: 0.5779 - acc: 0.8125 - val_loss: 0.6364 - val_acc: 0.7849

Epoch 00083: val_loss did not improve from 0.62755
Epoch 84/100
 - 3s - loss: 0.5759 - acc: 0.8142 - val_loss: 0.6368 - val_acc: 0.7900

Epoch 00084: val_loss did not improve from 0.62755
Epoch 85/100
 - 3s - loss: 0.5756 - acc: 0.8157 - val_loss: 0.6375 - val_acc: 0.7806

Epoch 00085: val_loss did not improve from 0.62755
Epoch 86/100
 - 3s - loss: 0.5728 - acc: 0.8167 - val_loss: 0.6361 - val_acc: 0.7918

Epoch 00086: val_loss did not improve from 0.62755
Epoch 87/100
 - 3s - loss: 0.5729 - acc: 0.8176 - val_loss: 0.6373 - val_acc: 0.7869

Epoch 00087: val_loss did not improve from 0.62755
Epoch 88/100
 - 3s - loss: 0.5706 - acc: 0.8198 - val_loss: 0.6388 - val_acc: 0.7819

Epoch 00088: val_loss did not improve from 0.62755
Epoch 89/100
 - 3s - loss: 0.5741 - acc: 0.8173 - val_loss: 0.6372 - val_acc: 0.7906

Epoch 00089: val_loss did not improve from 0.62755
Epoch 90/100
 - 3s - loss: 0.5722 - acc: 0.8178 - val_loss: 0.6344 - val_acc: 0.7888

Epoch 00090: val_loss did not improve from 0.62755
Epoch 91/100
 - 3s - loss: 0.5710 - acc: 0.8179 - val_loss: 0.6392 - val_acc: 0.7880

Epoch 00091: val_loss did not improve from 0.62755
Epoch 92/100
 - 3s - loss: 0.5701 - acc: 0.8190 - val_loss: 0.6339 - val_acc: 0.7886

Epoch 00092: val_loss did not improve from 0.62755
Epoch 93/100
 - 3s - loss: 0.5719 - acc: 0.8196 - val_loss: 0.6418 - val_acc: 0.7818

Epoch 00093: val_loss did not improve from 0.62755
Epoch 94/100
 - 3s - loss: 0.5719 - acc: 0.8165 - val_loss: 0.6316 - val_acc: 0.7914

Epoch 00094: val_loss did not improve from 0.62755
Epoch 95/100
 - 3s - loss: 0.5697 - acc: 0.8205 - val_loss: 0.6347 - val_acc: 0.7861

Epoch 00095: val_loss did not improve from 0.62755
Epoch 96/100
 - 3s - loss: 0.5694 - acc: 0.8189 - val_loss: 0.6400 - val_acc: 0.7903

Epoch 00096: val_loss did not improve from 0.62755
Epoch 97/100
 - 3s - loss: 0.5672 - acc: 0.8195 - val_loss: 0.6372 - val_acc: 0.7873

Epoch 00097: val_loss did not improve from 0.62755
Epoch 98/100
 - 3s - loss: 0.5682 - acc: 0.8186 - val_loss: 0.6356 - val_acc: 0.7891

Epoch 00098: val_loss did not improve from 0.62755
Epoch 99/100
 - 3s - loss: 0.5664 - acc: 0.8196 - val_loss: 0.6402 - val_acc: 0.7879

Epoch 00099: val_loss did not improve from 0.62755
Epoch 100/100
 - 3s - loss: 0.5688 - acc: 0.8189 - val_loss: 0.6349 - val_acc: 0.7900

Epoch 00100: val_loss did not improve from 0.62755

# Evaluating the model on the training and testing set
score = model.evaluate(X_train, y_train)
print("\n Training Accuracy on model_1:", score[1])
score = model.evaluate(X_test, y_test)
print("\n Testing Accuracy on model_1:", score[1])

40392/40392 [==============================] - 2s 58us/step

 Training Accuracy on model_1: 0.8096157654981184
8910/8910 [==============================] - 1s 57us/step

 Testing Accuracy on model_1: 0.7802469135534884

# Evaluating the model_1 on the training and testing set
score = model_1.evaluate(X_train, y_train)
print("\n Training Accuracy on model_1:", score[1])
score = model_1.evaluate(X_test, y_test)
print("\n Testing Accuracy on model_1:", score[1])

40392/40392 [==============================] - 3s 75us/step

 Training Accuracy on model_1: 0.8573232323232324
8910/8910 [==============================] - 1s 79us/step

 Testing Accuracy on model_1: 0.7858585858318273

# Evaluating the model_2 on the training and testing set
score = model_2.evaluate(X_train, y_train)
print("\n Training Accuracy on model_1:", score[1])
score = model_2.evaluate(X_test, y_test)
print("\n Testing Accuracy on model_1:", score[1])

40392/40392 [==============================] - 2s 58us/step

 Training Accuracy on model_1: 0.8356110120816003
8910/8910 [==============================] - 1s 57us/step

 Testing Accuracy on model_1: 0.7922558922291337

Conclusion¶

So from the above parameter tuning of the Deep NN , I have acheived an accuracy score of "79.2%". Almost 2% higher than the base models. This is a very open field for further tuning and depends highly on time and computational resources at hand. At the submission time (Sunday 6:00PM, 7th July 2019)for Axiata-Analytics for employment-test this was the best result I have acheived . Since I have already made entry into the www.drivendata.org , i will further tune and submit my predictions.¶

	basin_latitude	basin_longitude	latitude	longitude	basin_pump_dist
0	-11.670100	34.685700	-9.856322	34.938093	202.517704
1	1.000000	33.000000	-2.147466	34.698766	396.072413
2	4.574739	38.974731	-3.821329	37.460664	943.553742
3	-10.687900	36.263100	-11.155298	38.486161	248.454311
4	1.000000	33.000000	-1.825359	31.130847	375.346864

	amount_tsh	gps_height	population	years_elapsed	basin_pump_dist	funder_Action In A	funder_Grumeti	funder_Lottery Club	funder_Roman	funder_Unicef	installer_Artisan	installer_GRUMETI	installer_Roman	installer_UNICEF	installer_World vision	region_Iringa	region_Kagera	region_Manyara	region_Mara	region_Mtwara	public_meeting_True	public_meeting_miss_p_mtng	scheme_management_Other	scheme_management_VWC	scheme_management_miss_sch_mngt	permit_False	permit_True	extraction_type_gravity	extraction_type_submersible	management_other	management_vwc	management_wug	payment_never pay	payment_pay annually	payment_pay per bucket	water_quality_soft	quantity_dry	quantity_enough	quantity_insufficient	quantity_seasonal	source_dam	source_machine dbh	source_rainwater harvesting	source_spring	waterpoint_type_communal standpipe	waterpoint_type_communal standpipe multiple
0	6000.0	1390	109	12	202.517704	0	0	0	1	0	0	0	1	0	0	1	0	0	0	0	1	0	0	1	0	1	0	1	0	0	1	0	0	1	0	1	0	1	0	0	0	0	0	1	1	0
1	0.0	1399	280	3	396.072413	0	1	0	0	0	0	1	0	0	0	0	0	0	1	0	0	1	1	0	0	0	1	1	0	0	0	1	1	0	0	1	0	0	1	0	0	0	1	0	1	0
2	25.0	686	250	4	943.553742	0	0	1	0	0	0	0	0	0	1	0	0	1	0	0	1	0	0	1	0	0	1	1	0	0	1	0	0	0	1	1	0	1	0	0	1	0	0	0	0	1
3	0.0	263	58	27	248.454311	0	0	0	0	1	0	0	0	1	0	0	0	0	0	1	1	0	0	1	0	0	1	0	1	0	1	0	1	0	0	1	1	0	0	0	0	1	0	0	0	1
4	0.0	0	0	1	375.346864	1	0	0	0	0	1	0	0	0	0	0	1	0	0	0	1	0	0	0	1	0	1	1	0	1	0	0	1	0	0	1	0	0	0	1	0	0	1	0	1	0

	amount_tsh	gps_height	population	years_elapsed	basin_pump_dist	funder_Action In A	funder_Grumeti	funder_Lottery Club	funder_Roman	funder_Unicef	installer_Artisan	installer_GRUMETI	installer_Roman	installer_UNICEF	installer_World vision	region_Iringa	region_Kagera	region_Manyara	region_Mara	region_Mtwara	public_meeting_True	public_meeting_miss_p_mtng	scheme_management_Other	scheme_management_VWC	scheme_management_miss_sch_mngt	permit_False	permit_True	extraction_type_gravity	extraction_type_submersible	management_other	management_vwc	management_wug	payment_never pay	payment_pay annually	payment_pay per bucket	water_quality_soft	quantity_dry	quantity_enough	quantity_insufficient	quantity_seasonal	source_dam	source_machine dbh	source_rainwater harvesting	source_spring	waterpoint_type_communal standpipe	waterpoint_type_communal standpipe multiple
0	1.895665	1.041252	-0.150399	0.123433	-0.963565	0	0	0	1	0	0	0	1	0	0	1	0	0	0	0	1	0	0	1	0	1	0	1	0	0	1	0	0	1	0	1	0	1	0	0	0	0	0	1	1	0
1	-0.105970	1.054237	0.212290	-0.622187	-0.705447	0	1	0	0	0	0	1	0	0	0	0	0	0	1	0	0	1	1	0	0	0	1	1	0	0	0	1	1	0	0	1	0	0	1	0	0	0	1	0	1	0
2	-0.097630	0.025541	0.148660	-0.539340	0.024655	0	0	1	0	0	0	0	0	0	1	0	0	1	0	0	1	0	0	1	0	0	1	1	0	0	1	0	0	0	1	1	0	1	0	0	1	0	0	0	0	1
3	-0.105970	-0.584751	-0.258570	1.366133	-0.902306	0	0	0	0	1	0	0	0	1	0	0	0	0	0	1	1	0	0	1	0	0	1	0	1	0	1	0	1	0	0	1	1	0	0	0	0	1	0	0	0	1
4	-0.105970	-0.964200	-0.381587	-0.787880	-0.733086	1	0	0	0	0	1	0	0	0	0	0	1	0	0	0	1	0	0	0	1	0	1	1	0	1	0	0	1	0	0	1	0	0	0	1	0	0	1	0	1	0

	time	train_score	valid_score
SVC	1.000000	0.504158	0.872549
GaussianNB	0.000000	0.000000	0.000000
KNeighborsClassifier	0.000584	0.690730	1.000000
DecisionTreeClassifier	0.049523	1.000000	0.837908
RandomForestClassifier	0.027500	0.963715	0.977124
AdaBoostClassifier	0.138326	0.441746	0.757190
GradientBoostingClassifier	0.732052	0.566900	0.956536
ExtraTreesClassifier	0.005474	1.000000	0.956209
MLPClassifier	0.281464	0.748748	0.993464

		ward
region	lga
Arusha	Arusha Rural	1252
	Arusha Urban	63
	Karatu	326
	Longido	310
	Meru	1009
	Monduli	189
	Ngorongoro	201
Dar es Salaam	Ilala	497
	Kinondoni	93
	Temeke	215
Dodoma	Bahi	224
	Chamwino	347
	Dodoma Urban	358
	Kondoa	523
	Kongwa	361
	Mpwapwa	388
Iringa	Iringa Rural	728
	Kilolo	349
	Ludewa	564
	Makete	630
	Mufindi	520
	Njombe	2503
Kagera	Biharamulo	403
	Bukoba Rural	487
	Bukoba Urban	88
	Chato	236
	Karagwe	771
	Misenyi	260
	Muleba	402
	Ngara	669
Kigoma	Kasulu	1047
	Kibondo	874
	Kigoma Rural	824
	Kigoma Urban	71
Kilimanjaro	Hai	625
	Moshi Rural	1251
	Moshi Urban	79
	Mwanga	519
	Rombo	594
	Same	877
	Siha	434
Lindi	Kilwa	392
	Lindi Rural	388
	Lindi Urban	21
	Liwale	154
	Nachingwea	300
	Ruangwa	291
Manyara	Babati	511
	Hanang	274
	Kiteto	193
	Mbulu	297
	Simanjiro	308
Mara	Bunda	438
	Musoma Rural	396
	Rorya	210
	Serengeti	716
	Tarime	209
Mbeya	Chunya	298
	Ileje	231
	Kyela	859
	Mbarali	626
	Mbeya Rural	485
	Mbozi	1034
	Rungwe	1106
Morogoro	Kilombero	959
	Kilosa	1094
	Morogoro Rural	521
	Morogoro Urban	96
	Mvomero	671
	Ulanga	665
Mtwara	Masasi	528
	Mtwara Rural	423
	Mtwara Urban	124
	Nanyumbu	158
	Newala	231
	Tandahimba	266
Mwanza	Geita	488
	Ilemela	142
	Kwimba	627
	Magu	824
	Missungwi	348
	Nyamagana	1
	Sengerema	331
	Ukerewe	341
Pwani	Bagamoyo	997
	Kibaha	269
	Kisarawe	223
	Mafia	132
	Mkuranga	560
	Rufiji	454
Rukwa	Mpanda	679
	Nkasi	428
	Sumbawanga Rural	521
	Sumbawanga Urban	180
Ruvuma	Mbinga	750
	Namtumbo	694
	Songea Rural	693
	Songea Urban	80
	Tunduru	423
Shinyanga	Bariadi	1177
	Bukombe	514
	Kahama	836
	Kishapu	399
	Maswa	809
	Meatu	468
	Shinyanga Rural	588
	Shinyanga Urban	191
Singida	Iramba	544
	Manyoni	377
	Singida Rural	995
	Singida Urban	177
Tabora	Igunga	338
	Nzega	575
	Sikonge	170
	Tabora Urban	155
	Urambo	382
	Uyui	339
Tanga	Handeni	254
	Kilindi	161
	Korogwe	412
	Lushoto	694
	Mkinga	288
	Muheza	334
	Pangani	305
	Tanga	99

	date_recorded	year_recorded
0	2011-03-14	2011
1	2013-03-06	2013
2	2013-02-25	2013
3	2013-01-28	2013
4	2011-07-13	2011

	date_recorded	construction_year	year_recorded
0	2011-03-14	1999	2011
1	2013-03-06	2010	2013
2	2013-02-25	2009	2013
3	2013-01-28	1986	2013
4	2011-07-13	0	2011

	id	amount_tsh	date_recorded	funder	gps_height	installer	longitude	latitude	wpt_name	basin	subvillage	region	region_code	district_code	lga	ward	population	public_meeting	recorded_by	scheme_management	scheme_name	permit	construction_year	extraction_type	extraction_type_group	extraction_type_class	management	management_group	payment	payment_type	water_quality	quality_group	quantity	quantity_group	source	source_type	source_class	waterpoint_type	waterpoint_type_group	year_recorded	month_recorded	years_elapsed	const_yr_impt_2010
0	69572	6000.0	2011-03-14	Roman	1390	Roman	34.938093	-9.856322	none	Lake Nyasa	Mnyusi B	Iringa	11	5	Ludewa	Mundindi	109	True	GeoData Consultants Ltd	VWC	Roman	False	1999	gravity	gravity	gravity	vwc	user-group	pay annually	annually	soft	good	enough	enough	spring	spring	groundwater	communal standpipe	communal standpipe	2011	3	12	1999
1	8776	0.0	2013-03-06	Grumeti	1399	GRUMETI	34.698766	-2.147466	Zahanati	Lake Victoria	Nyamara	Mara	20	2	Serengeti	Natta	280	NaN	GeoData Consultants Ltd	Other	NaN	True	2010	gravity	gravity	gravity	wug	user-group	never pay	never pay	soft	good	insufficient	insufficient	rainwater harvesting	rainwater harvesting	surface	communal standpipe	communal standpipe	2013	3	3	2010
2	34310	25.0	2013-02-25	Lottery Club	686	World vision	37.460664	-3.821329	Kwa Mahundi	Pangani	Majengo	Manyara	21	4	Simanjiro	Ngorika	250	True	GeoData Consultants Ltd	VWC	Nyumba ya mungu pipe scheme	True	2009	gravity	gravity	gravity	vwc	user-group	pay per bucket	per bucket	soft	good	enough	enough	dam	dam	surface	communal standpipe multiple	communal standpipe	2013	2	4	2009
3	67743	0.0	2013-01-28	Unicef	263	UNICEF	38.486161	-11.155298	Zahanati Ya Nanyumbu	Ruvuma / Southern Coast	Mahakamani	Mtwara	90	63	Nanyumbu	Nanyumbu	58	True	GeoData Consultants Ltd	VWC	NaN	True	1986	submersible	submersible	submersible	vwc	user-group	never pay	never pay	soft	good	dry	dry	machine dbh	borehole	groundwater	communal standpipe multiple	communal standpipe	2013	1	27	1986
4	19728	0.0	2011-07-13	Action In A	0	Artisan	31.130847	-1.825359	Shuleni	Lake Victoria	Kyanyamisa	Kagera	18	1	Karagwe	Nyakasimbi	0	True	GeoData Consultants Ltd	NaN	NaN	True	0	gravity	gravity	gravity	other	other	never pay	never pay	soft	good	seasonal	seasonal	rainwater harvesting	rainwater harvesting	surface	communal standpipe	communal standpipe	2011	7	1	2010
5	9944	20.0	2011-03-13	Mkinga Distric Coun	0	DWE	39.172796	-4.765587	Tajiri	Pangani	Moa/Mwereme	Tanga	4	8	Mkinga	Moa	1	True	GeoData Consultants Ltd	VWC	Zingibali	True	2009	submersible	submersible	submersible	vwc	user-group	pay per bucket	per bucket	salty	salty	enough	enough	other	other	unknown	communal standpipe multiple	communal standpipe	2011	3	2	2009
6	19816	0.0	2012-10-01	Dwsp	0	DWSP	33.362410	-3.766365	Kwa Ngomho	Internal	Ishinabulandi	Shinyanga	17	3	Shinyanga Rural	Samuye	0	True	GeoData Consultants Ltd	VWC	NaN	True	0	swn 80	swn 80	handpump	vwc	user-group	never pay	never pay	soft	good	enough	enough	machine dbh	borehole	groundwater	hand pump	hand pump	2012	10	2	2010
7	54551	0.0	2012-10-09	Rwssp	0	DWE	32.620617	-4.226198	Tushirikiane	Lake Tanganyika	Nyawishi Center	Shinyanga	17	3	Kahama	Chambo	0	True	GeoData Consultants Ltd	NaN	NaN	True	0	nira/tanira	nira/tanira	handpump	wug	user-group	unknown	unknown	milky	milky	enough	enough	shallow well	shallow well	groundwater	hand pump	hand pump	2012	10	2	2010
8	53934	0.0	2012-11-03	Wateraid	0	Water Aid	32.711100	-5.146712	Kwa Ramadhan Musa	Lake Tanganyika	Imalauduki	Tabora	14	6	Tabora Urban	Itetemia	0	True	GeoData Consultants Ltd	VWC	NaN	True	0	india mark ii	india mark ii	handpump	vwc	user-group	never pay	never pay	salty	salty	seasonal	seasonal	machine dbh	borehole	groundwater	hand pump	hand pump	2012	11	2	2010
9	46144	0.0	2011-08-03	Isingiro Ho	0	Artisan	30.626991	-1.257051	Kwapeto	Lake Victoria	Mkonomre	Kagera	18	1	Karagwe	Kaisho	0	True	GeoData Consultants Ltd	NaN	NaN	True	0	nira/tanira	nira/tanira	handpump	vwc	user-group	never pay	never pay	soft	good	enough	enough	shallow well	shallow well	groundwater	hand pump	hand pump	2011	8	1	2010

	id	status_group
0	69572	functional
1	8776	functional
2	34310	functional
3	67743	non functional
4	19728	functional

	time	train_score	valid_score
SVC	229.135942	0.737621	0.737077
GaussianNB	0.090633	0.473435	0.472668
KNeighborsClassifier	0.224467	0.835388	0.775698
DecisionTreeClassifier	11.433602	0.997450	0.726580
RandomForestClassifier	6.389280	0.978436	0.768766
AdaBoostClassifier	31.773513	0.704917	0.702119
GradientBoostingClassifier	167.763767	0.770499	0.762527
ExtraTreesClassifier	1.344342	0.997450	0.762428
MLPClassifier	64.558635	0.865790	0.773718